Arlong never hurt any of his Fishmen minions, only Nami, who he considered a "sub-human" (lol you get the idea) slave. Luffy would hate Arlong much more than Crocodile, simply because the suffering Arlong inflicted on Nami was greater than what Crododile did to Vivi. Thats not to say he would forgive Crocodile, but its obvious he would hate Arlong more.

Speaking of Arlong, Jinbei and Arlong used to be in the same crew, before it split, and Arlong was said to be Jinbei's equal or something. If Luffy does get down there to where Crocodile/Ace/Jinbei are, seeing what goes down between those two will be interesting too.