Kogami will hunt him back in the end, but I'll think the old guy will give him a pretty hard time. He's a seasoned hunter, after all.. though the quality of his prey is questionable. Plus, he seems to release those cybernetic dogs instead of doing some real hunting.. so I have no idea where he gets this "thrill" from.
If it was instead the excitement from holding something made from taboo human parts as trophies, that's more believable.
As for the whole guarding-psycho-pass idea, I think it's just Gino's take on things. If it was actually something Inspectors were supposed to keep in mind, it would have been part of the basic training even Akane received. The training modules might have programmed the Inspectors indirectly by explaining that Inspectors are only interested in results but not the how of a crime, but even that's a guess at best.