going from A all the way to D strikes me as somewhat extreme for one who is used to the Pass. Why not look into a good A/B first?
The main reason AB exists is to reduce heat and power consumption, with some of the benefit of class A operation.
Class D allows a good designer to sidestep some of the reasons for class A operation in the first place- a classic example being that of crossover distortion. Most class D amplifiers are literally incapable of this kind of distortion. In addition, the distortion of the output section isn't made in the same way that it is in a class A or AB design. The result can be that the distortion the class D amp generates can be far more benign to the ear.
I've heard class D amps that compare more than just favorably to a Pass Labs amplifier. These days its not about class of operation, it simply about the sound.