Can't really blame them. That's what easy life with unlimited riches does to people. These guys have also the unfortunate extra of serving under an (ex-)celestrial dragon. Just recall how stupid and deranged those guys looked back during the Shabondy arc. There's no saving them.
Law had quite an unfortunate past, for sure. Flamingo shouldn't be saying anything to him about his own past misfortune. It can hardly be worse, except in his own opinion, of course.