Naruto attacked them. Not that it should be a topic of discussion or debate. It is that simple.

I doubt the reaction was from the sword gesture, either. The broad said Sasuke. That is all it takes.