Great thread; thanks, ghdprentice! I’ve got several things to say here.
First, I completely agree that ambient noise is almost surely a far bigger problem for most high-end systems than any less-than-optimal component. It’s at least as important as room acoustics and proper speaker positioning. In my previous house, when I had to buy a new fridge, my first criterion: how quiet the damn thing was. This always amused salesmen. Now, I’m lucky; the house is big, the kitchen is two rooms (and walls) away, and my listening room is very quiet. So quiet, that the slight hum my old amplifier’s transformer has developed bugs me. I’ve tried many strategies to resolve this, some of them discussed, or learned about, on this forum (e.g., a varnish "dip-and-bake"; you don’t want to know). Like curiousjim, I also took my unit to a very competent tech, and he couldn’t hear the hum at all. Of course, his shop was on a busy street. But I’m fanatical, I’ll admit, about creating as quiet a space as possible for listening. I think it was ghdprentice who mentioned watching a fire while listening. I’ve got a fireplace in my listening room, too, but I never do any serious listening with a fire going. The fire makes too much noise!
I’ve got a John Cage story that’s relevant here. I met him once, through my wife. He told me about having spent an hour in an anechoic chamber. He claimed that, even in "complete silence," he heard two faint sounds, one very low, one very high. He concluded the low tone was the sound of his blood coursing through his body, and the high hiss-like tone was the sound of his neural activity. But I think he just suffered from tinnitus without realizing it; there’s no way one can "hear" one’s neurons firing as a constant high-pitched hiss! Of course, John Cage was all about the impossibility of "silence," about how "music" is more a matter of the attention of the listener than the intention of a composer, so his story had a Cagian moral.
As for SPL meters, on Apple products or whatever else—you know, I suppose, that they must be properly "weighted." Most inexpensive SPL meters are "A" weighted, and meant for testing ambient noise at worksites that may be dangerous. "A" weighting cuts out at 100 Hz, if I remember correctly; it’s somewhere below which there is A LOT of musical information. If you have a meter that can switch between "C" and "A" weighting, it’s startling to do so; music that reads, say, 80 db in "A" will jump to 90 db or more in "C." And, of course, Apple products don’t prioritize the noise metering they throw into their devices. Get a proper meter if you want an accurate reading.
Regarding panzewagn’s comment about architects using HVAC and other drone noises to "create some sense of privacy"—yes, of course this seems crazy, but it may not be not quite as pointless as it seems. My dad was a Caltech scientist, and a very finicky person. We lived at the end of a private road up against the San Gabriel mountains, where it was very quiet—but not quiet enough for him! In his study, he had an old heater with a fan; the heater didn’t work, but the fan did, and he ran it all the time when he was working. He claimed that the constant droning sound distracted him from being disturbed by the occasional dog bark or distant car horn or whatever. I think I get this.
One last thing. There’s a psychoacoustic phenomenon—I forget what it’s called, but there’s a Wikipedia page about it—according to which certain low-level kinds of noise actually focus the mind’s attention on the "meaningful" sounds those noises might otherwise seem to mask or distract from. There is a theory, which makes a lot of sense to me, that this phenomenon explains why some people prefer vinyl to digital: the surface noise and the occasional pop and click, far from distracting from the music, actually provide a background against which conscious attention isolates the musical information, highlights it as it were. This could explain my dad’s practice mentioned above, but would be diametrically opposite to the preponderance of views expressed here (including my own) that serious listening demands as quiet a "noise floor" as possible. It’s one of the reasons I don’t buy into the vinyl fetish; there’s just too much scientific evidence of its irrationality, and in any case, it certainly doesn’t correspond with my own listening preferences. Most of us live in perpetually noisy environments; it is possible that, for those who do, brain "circuitry" has accommodated itself to attention against a constant background of noise, and this may be why vinyl sounds better to them. I’m lucky in that regard to live almost all the time in environments that are very close to silent.