As it stands now the switching frequency noise output filter, has phase shift effects that reach right down to 5khz, that doubled for the worse at 10khz and doubled worse again 20khz. That's what many listeners find objectionable, in the upper mids and highs.This statement is false. The filter is set to be operational at a fairly high frequency, and are usually 12 db per octave filters. 6 db/octave filters exhibit phase shift over the widest range, so if the pole frequency is at 90KHz you could see some artifact at 9KHz but not at 5KHz. But most filters I've seen are 12db/octave and so don't have phase shift artifact anywhere within the audio passband.
What many listeners likely 'find objectionable' is not the filter- its likely something else. Several issues exist that can account for that- higher ordered harmonics caused by poor loop feedback implementation, breakdown of the encoding scheme at higher volume levels, distortion from the input circuit... but its not the filter. IME a good class D amp has artifacts, but not the kind typical to traditional solid state, so in many cases I find especially the later generations to be more musical than traditional solid state.