"George, which amps have you experienced which are the audible cause of your pessimism?"
Many and they all have the same trait, explained below.
Until the future technology gives us higher switching frequency, we'll get this (linked) switching frequency noise on the top of the square wave in pic 2 and that's at 1khz!!!
At 10khz (not shown) it's 10 x worse (if this was a linear amp, you'd say it was broken)
And the output filter is already started to roll the audio frequency off at 10khz to try to minimize it Pic 1.
http://www.stereophile.com/content/theta-digital-prometheus-monoblock-power-amplifier-measurements#d5BMlUSvHXmmeEbA.97
Today we have two detrimental ways to get rid of the Class D switching noise, with output filters on these amps.
1: The filter to roll off earlier into the audio band to minimise the noise, result (soft/opaque/dark treble and upper mids and hf phase shifts)
2: The filter to roll off later with no treble roll off and minimized hf phase shift, but let 10 x the switching noise through (hard in your face treble and upper mids).
Or we bite the bullet and use Class D just for bass, and wait for technology to advance to get much high switching frequencies for class D so we can then filter them out much higher and more aggressively with being near the audio band.
It will happen one day and that's when I will get one as well, but today it is just not ready yet, it's still developing, when it does our heavy massive linear amps will become boat good anchors.
Cheers George