That might apply to lots of one episode wonder characters, but with important characters there have been plenty of good looking villains. I think there is, though, some merit in doing it like this. Ugliness is either external or internal, and by combining them, Oda simply cuts a corner in needing to spend more time to make evident the internal ugliness. If we consider Boa's words, a really beautiful person can do much more yet still retain sympathy. An ugly one will rather need to prove they aren't a monster, so a short scene will do fine to establish an ugly one as a true villain.
More importantly, it was interesting to see Dragon. Seems like he already didn't do much back then, just like he isn't doing much in the later timeline. At least much of anything we would have seen any trace of.