Somehow I think that if somebody in a neighbouring room can be disturbed by the noise from your headphones, you should be already far more worried about your own accumulating hearing loss.

Sometimes I take my headphones off and forget about the music playing and after a while I start to wonder about the vague, barely audible sounds eminating from somewhere before I realise they come from half a meter away from the headphones lying on the table. That's the kind of volume I consider normal. From the other side of my room I wouldn't have a chance of detecting the sound.