Nobody can hear 500khz, it the filter that has to deal with it and what’s left over and the byproducts of it that is the problem.
What is left over is 500kHz. Byproducts are possible (modulation), assuming nonlinearity of motion, if tweeter can move membrane at 500kHz - no chance.
Filter or frequency have very little to do with quality of the sound, IMHO. It is related to quality of the modulator, slew rate of output devices (accuracy of the duty cycle), ringing caused by inductance and capacitance of output devices etc. Actually, increasing switching frequency can make it worse because it will reduce accuracy of the duty cycle - calling for deeper NFB. As I showed before (using 50kHz Lamm amp as an example) phase delay in audio band has nothing to do with extension and, if anything, might be beneficial since most of the speakers accelerate phase (much more) at these frequencies.
Also, increasing switching frequency will reduce amount of residue on the speaker cable but at the same time it will make it radiate much better.