The land occupied by humans is actually helluva large to be just a single walled off place, like Ryll indicated up there with his comparisons.

From the center (king's palace?) 480km to the outer wall (that now has a single breach). However, assuming the outer wall is roughly circular, it's fricking 3000km long! The map also shows those smaller walled off areas. Maybe they are the bigger towns, while the rest is countryside, villages, forest, etc.

To be honest, I think the mangaka made a terrible mistake with these numbers. Unless there was a period of hundreds of years of a slow, losing battle against the giants and humans realised their doom from early on, plus the giants weren't everywhere equally. So that when people elsewhere were being eaten and eradicated, the people roughly around where this city stands, plus any lucky refugees, had several generations to build these massive structures relatively unhindered. Let's not forget they don't seem to have much in the way of heavy machinery, relying on horses and such, so building thousands of kilometers of dozens of meters high walls is a godly accomplishment.