Like you assumed, the sonic signature of the amp has more to do with the surrounding circuits as well as the capability of the power supply. However, there are some aspects to the type of transistor and number.
Generally speaking, a FET type transistor (like JFET or MOSFET) will typically have more of a warm/rich character. FET transistor are usually not precise and require a resistor or pot to balance for DC offset (when used in an input stage).
A bipolar transistor will, typically, sound more clean and will generally have lower distortion. Bipolar are manufactured more precise and don’t need DC offset adjustment. A bipolar transistor will also have more input bias current (which means there can be DC on the output).
The number of output transistor an amp uses can dictate how well it moves the speaker. For example, when you compare an amp with 4 output transistors (2 for positive waveform and 2 for negative waveform) with an amplifier that has 12 output transistors, the transistors on the 12 amp will work 1/3 as hard for the amp to push the same amount of current to the speaker. Each of the 12 transistors do not have to slew up the output as much when compared to the 4 transistor amp). As long as the power supply is capable, the 12-transistors will be more efficient in having more current on tap for the demanding music passages (even at low volume).
When having a large number of transistors, you also get into the linearity issue. A ultra-high end company like D’Agostino will hand match all the output transistors so that they are as close as possible - each one pushing through the same amount of current for a waveform. Most of the amplifiers are machine manufactured and do not have matched transistors. For example, you have 4 transistors for positive waveform -- these transistors could each be pushing different voltages for a given waveform (i.e. 1.7V, 1.88V, 1.65V, 1.75V). While not a bad thing overall, the accuracy of the matched transistors is probably more linear.