I think this is a comedy at heart, and the reason it doesn't get away with abusing the characters as much with the audience (us) is that

A. In series with a similar premise (i.e. Konosuba), all of the characters get some comeuppances for us to laugh at. Here, it is only MC who is constantly getting the short end of the stick.

B. Usually all of the characters are flawed in some way so we can get behind their retribution or enjoy their machinations, but here, the main character may be a little conceited, but is a generally pleasant person who isn't actively throwing others under the bus to improve his own standing. He's just a punching bag for this world, and that setup gets old after a while.