Also, Robin had seastone cuffs for an entire arc and it never really seemed to make her physically weak. Same with Ace. He's able to walk under his own power and everything.

Maybe the cuffs are just made differently.