As I understand it, "noise floor" does not really refer to what the OP was asking about, which might better be described as "ambient noise." Millercarbon's comment is relevant to the concept of "noise floor," which refers to the capacity of one's system for resolving very subtle sounds--not just quiet sounds, but subtle sounds. The decay of a resonant acoustic after the performers stop playing. The difference in timbre between an oboe and an English horn. These subtleties are easier to perceive in a quiet environment, but a good system will resolve them in thrillingly audible ways even when there is a plenty of measurable ambient noise (for instance, surface noise from vinyl).
In fact, there is a theory in psychoacoustics that a certain amount of low-level noise actually enables the brain to distinguish subtle differences better than complete silence would. I'm not at home at the moment, and can't tell you the name of this theory (I don't have my main laptop with me here in Dalmatia), but I think it's a plausible explanation for why so many people find vinyl to sound "better" than a digital signal that measures better.
That being said, the OP's question is still extremely important to happy audiophilia, IMO. Before we moved to our current house, which is on a mountaintop in the country and has a listening room with gorgeous acoustics (and ambient noise in the low 20db range), we lived in a house with a "great room" which shared the same space between the living room, where the stereo was, and the kitchen, where the refrigerator was. I eventually went shopping for a new fridge--it's being quiet the only important criterion. The salespeople were amused to be told that replacing my fridge would be the single best improvement I could make to my stereo system. And even now, I sometimes turn the HVAC system off when seriously listening. We have a very good, and very quiet HVAC system, but I still notice every time it comes on, and I can be distracted by it during quiet passages of music.
Of course ambient noises are distracting! Of course one would wish to limit them as much as possible. Serious lovers of "classical" music disdain the Hollywood Bowl, even though it's such a crowd pleasing venue: distant freeway noise, planes randomly flying overhead, etc. etc. There's no way to control an outdoor acoustic in a big city. Even the Ojai Music Festival, as wonderful as it is, has to deal with that problem, and is inevitably compromised by it. This, it seems to me, is what the OP is asking about. And yes, it makes perfect sense to control the extraneous noises one can control in one's environment. Hurray for quiet refrigerators!
