It's absolutely the second one. He's not only lamenting that all the people he really wants to beat (Arata, Nikuman) are in A-Class, but also that he can't "defend" Chihaya while they are separated. Arata basically has free reign over Chihaya's heart right now. Normally, I would say karuta has free reign over Chihaya's heart, but since karuta has always been the vehicle that reunites her with Arata...
Of course, Chihaya has her own rival of both karuta and love to worry about. Shinobu peering over the table and then visibly pissed she didn't get to play Arata right away (and subsequently not giving a shit about who her opponent was after that) was amusing. Not to mention the two of them are basically on the same wavelength when it comes to personal taste. I also love that people are still talking about her weight loss.
What does annoy me is, as Kraco mentioned, Chihaya always has to be cast as an underdog somehow. It's very disappointing that she never really gets to shine lately. They allow her team to win the tournament, and now suddenly she has no chance to win or even get very far into the individual tournament. Last time, fainting ruined their chances as the team tournament. For every win, Chihaya has to lose just as much on one of her other goals. Worse, she's unlikely to proceed to the second round since she's paired up with the Takarazuka girl. She might win by emulating Arata and Shinobu's 'middle finger exactly on the edge' strikes, but it will feel cheap if she wins either way.