find out whether the phase shift has any relevance in your listening.
If you believe there nothing to hear when you have 70 degrees phase shift at 5khz and still 50 degrees at 2khz!! then there’s no hope for you.
https://ibb.co/JdrgGBH
I have heard class D amps that sound euphonic/rolled off in HF, and others that sound harsh.
All because of the setting of the switching frequency output filter and where it acts, either "too high" or "too low"
"rolled off in HF" (too low) , and "others that sound harsh" (too high), with today’s 500khz switching frequency.
Raise that switching frequency 3 x to 1.5mhz as you can with GaN, and you’ve got rid of the problem because the output filter can then also be raised 3 x higher, and then you don’t get that phase shift so low down into the audio band or the problem of as you say ("rolled off in HF, and others that sound harsh"), because the filter then can be set to do it’s job fully and rid the switching frequency completely and not effect the audio band with phase shift.
It’s that simple, even the most technically inept should be able to understand, maybe! Or you can believe the ones that are trying to tell you it doesn’t matter. From day one Class-D has had this problem, because of it’s low switching frequency/filter.