That "tube sound" and power ratings

This might be a newbie question since I've only begun researching tube technology. I understand to some degree the theory that tube sound is partly related to second harmonic distortion vs. the more prevalent odd order harmonic characteristics of SS. If "tubies" prefer that sound (I might be one of them), does it make sense to carefully match an amplifier's power rating such that it is NOT TOO HIGH for the speakers it's driving? If the rating is too high won't that mean lower distortion and hence less tube sound for a given volume for those speakers than a lower power tube amp (in general that is - I realize not all Watts are the same). So won't a high wattage tube amp have less of the special tube sound "tubies" like at their preferred listening volume?

I realize I'm likely missing something here. Set me straight!