The XA.5s are sweeter, darker, more liquid, palpable and mellow/relaxing. The X.5s are livelier, faster/snappier, subjectively more dynamic and punchy. In terms of bass the XA is warmer and more diffuse (but not untight or lack control) and the X is leaner and tighter.
I'm speaking purely in relative terms as all Pass amps are sweet, dynamic, etc compared to the competition.
I like the XA for jazz, classic rock, etc and the X for massive orchestral or choral works, not so much because of reasons of power (the big XAs have plenty) but because that's how they are most tonally suited.