Sort of. I am concerned that foreign people will visit Japan on vacation, grow to love it, then decide to move there, bringing possible bad influences with them, and altering the people that they meet, for better or worse. That's the key, it doesn't necessarily have to be a "bad" change. But what's good for some is bad for others.
This doesn't just apply to the Japanese, but all cultures. We just happen to be talking about the Japanese because of their unique circumstances and influence on the rest of the world that somehow makes people want to go there.