Is it even really so? Japan, like any old industrialised, service heavy countries is full of jobs that don't stick to the traditional office hours. There are endlessly people who must accept evening and night shifts at work. Those bastards whispering about his unemployment are also using services that require people to work outside of their primary concept of working hours. So, are those people then supposed to sit indoors all day long until they go to work? If he wasn't a wanted man, he should have educated them.

But then again, I reckon most people do realise this and those women were merely regular idiots.