We have hardly yet seen all there could be to see on that transvestite island. For all we know, some really tough individual could be living there. The same goes for the bird island although I imagine Chopper's lessons are of some other sort.

In any case Luffy and Nami landing on such extremely relevant and convenient islands is too much to be a coincidence.