Those days of really loud are long gone. As a teen, parents would regularly complain that they could hear my headphones downstairs.
Now, I find the bulk of my listening is around 75-80 db. Most music can come through nicely at that level, and if listening to solo cello, or unaccompanied vocals, it doesn't seem out of place at that level. Having said that, I was recently listening to some Bach harpsichord at at friends while "testing" speakers at 110db. I made it clear that if the police came, I was going to blame it all on my friend. No way I was going to have a complaint levied at me for listening to that at those levels...
When it's really late and I don't want to disturb anyone, the volume drops a little. But the room is in the basement, and bedrooms on third floor.
And it is has been stated above, volume is very music dependent. Big orchestral pieces, or metal do come across "better" when listened to at appropriate volumes. (Harpsichord being the obvious exception...)
And, in the room as it stands now, anything above 95 to 100db, gets rough sounding as the room is being pressurized to the point of noise.
Now that I think of it, if I want to listen really loud, so I can have the physical impact of the music, suppose I could wear some hearing protection - like I would a a concert. That would be something if the neighbours complain, police show up and I have to explain the logic of my music listening approach...