Hm... Zabuza was pretty strong... Made Kakashi use the Sharingan... there was also another legendary ninja, but (s)he hasn't been formally introduced. I personally think (s)he is the Genin that was with Oro and Ero-sennin at the beggining. The blonde chick. Thats just a guess tho.

So My list (in order)-

Legendary-
Jiraiya
Oro

Konoha-
Kakashi
Gai
Zabazu (I know he isn't hidden leaf, but he's strong, and I didn't want to make a Hidden Mist for 1 character)