I wish you could 'leave the speakers out of it'.
Sorry, no can do. What you get as sound is not just 'amp' or 'speaker', but the combo. Happy combo makes good sound. mating a speaker with hi impedance swings and huge phase angles on even the best tube amp will probably result in unhappy. Likewise, a good, well executed SDFR design may be happier with tubes.
It's all about that mystical term....'synergy'.