In Mikasa's defence, the gaps in his armor appeared after he shed it off.

I think the titan was basically supposed to be invulnerable to swords before that.. or at least nothing vital/important could be reached.