Their passive defense is a product of the fact that their bodies have changed "into" their element. Blackbeard's darkness absorbs everything. Crocodile's sand is just plain old sand, and "explodes off" if he is struck hard. It clumps together if it gets wet. Just plain old sand, under Crocodile's control.

Luffy's rubber is just plain old rubber, and bounces if it is struck. It gets cut by blades, just like plain old rubber. That is to say, Luffy has a form of this passive defense. It just isn't as awesome as Enel's (who basically has no weakness aside from the obvious one, since his element has none/few) and significantly better than Blackbeard's. Someone who was literally made of diamond would have it too, and theirs would presumably be stronger than Luffy's. Someone who was made of glass would have it, but it would be a terrible weakness unless they learned to stay in a liquid state (in which case, they would have the flowing action described)