You would hate this then, because it is 100% that. Even though it's more nuanced than the usual "Dumb protag cares too much about fighting to notice girl likes him."
It makes more sense though as a middle-aged guy why he wouldn't think he has a shot.
He doesn't realize that he's cool, and thinks he's too old. So he just assumes there's no way they can be interested in him, and that they just view him as their teacher.
It doesn't help that they're not straightforward about their feelings, and when they hint at it, he just brushes it off as they're joking around with him.