Would he need to put it into words? We already know he considers her beautiful and attractive (which is a big deal for him, due to personal reasons), he admires her personality, energy, and social skills, he fully shares her respect for other people's varied hobbies and interests, he has a whole lot of fun just being in her company and doing mundane stuf with her, and, like you said, he's sexually attracted to her. He has more going on for a love than great many characters in myriad series. If he doesn't seem to be clearly in love with her, I'd say it's because he's really shy and wouldn't dare to dream of making a girlfriend out of the popular class beauty, as he considers himself almost an outcast or at the very least an invisible loner.