Geoff, not everything is black and white. Systems do not go from "nice sounding" to "intolerable distortion" over a 1 db range. It is a gradual thing. But the ear does tend to suddenly notice things when they get to a certain threshold, just as the eye does....we don't sense things linearly.
In the last 20 or 25 years, it has certainly not been a huge priority of high end to get extremely loud playback at low distortion levels. Producers have been fixated on imaging, pretty female vocals, lots of deep bass, and maybe above all "high definition" or lots of detail, because reviewers and audiophiles fixated on that stuff, especially the detail.
Remember, most of the folks spending the big bucks are heavily influenced by the reviews and manufacturer hype. I think most of the expensive speakers have plenty of problems, the least of which is rising distortion with louder playback levels. Pricing has gone crazy, but sonic quality has not kept pace on a linear level with pricing.