Personally I have lost all interest in hard to drive speakers because I find their total cost of ownership just too high for little value.+1 on that!
When you can connect headphones to the power supply of any amplifier and hear the audio signal, you may count on the fact that the power supply is contributing to IMD. This is a very audible form of distortion! It is for this reason that we have a separate power supply for our driver circuits as opposed to our output circuits.
IMO this should be the case in a class D amp too, and I think that should pan out as the input circuitry usually has very different requirements as opposed to the output section.