What contributes most to a change in how an amplifier sounds?

Okay, with so many opinions it seems no one here actually knows the answer to the question. Uh boy!

I gave the answer..., output impedience.

The frequency response of an amplifier will vary with the output impedience when pushing a real life speaker load, and not some fixed value resistor used to determine the power.

The lower the output impedience, the flatter the frequency response across the band. An amp with a high output impedience may vary by several decibels across the band when pushing a real life load.  You can't get around Ohm's Law.

You will hear a difference of several decibels across the band far more than the difference between say 0.01% and 0.001% distortion, or...

Pretty sure I nailed it on the first post.

@emergingsoul The secondary thing is the distortion signature of the amp, which audiophiles call the 'sonic' signature. This is in turn caused by something called the 'transfer characteristic' of the amp. The transfer characteristic has something to say about how the amp responds to transients, in particular those that overload the amp. It also says what sort of distortion will be produced, for example if based on a quadratic exponent, the 2nd harmonic or a cubic exponent, resulting in a dominant 3rd harmonic.

These two harmonics are treated by the ear much the same way in that they are innocuous. But intermodulation distortion is not and also plays an important role that is defined by the transfer function.

If you want to know more about this, read this article which starts at page 35 at the link. It might be more than you want to know, but it does answer your question correctly.

Pretty sure I nailed it on the first post.

What!?  Not the loudness button??