I certainly agree that amplifiers and recordings make significant differences. To those I would add cables, followed by footers and many 'tweaks'. And synergy can often make a lovely difference.
When I have discussed this topic of musicality with other professionals, the following have always been common themes:
Above all, speakers must first be setup properly in a room possessing at least 'OK' acoustics. If someone has mentioned this above, my apologies. When a room's acoustics are 'OK' and the room is larger than a spare bedroom, I do know that, if finding the right placement for speakers and chair remains difficult (even for a reviewer), it is wise to suspect the speakers.
You might think that the 'best systems' would make less-than-perfect recordings unbearable in at least some ways. Yet I and others have heard countless 'best systems' make most every recording wonderfully musical, completely entertaining, bodily-moving, and emotionally engaging. There was no attention paid to 'harshness' or 'distortion', nor even to the details of 'soundstage', 'airiness' and the like. It was always the performance that came to life.
Now, experienced recording engineers, producers and artists always speak of the need to capture the performance when a band is on fire!, and how this outweighs any recording quality. They go on to say we thus have every reason to expect the recordings of major artists, even when made in the 1920's, to have captured the performers in full stride. And when you don't feel that, it is the fault of playback, not recording.
Returning to speaker performance, it is important to note, in no particular order, that there are many mechanical flaws in otherwise high-tech appearing woofers and tweeters that keep us from making emotional connection to the music. These are measurable.
There can be many acoustic problems from how the air is moving right near any cone or dome, both in front and behind, and by how the air is allowed to reverberate in the chamber or cabinet behind. All of these can be measured, some indirectly.
There are many reasons that the 'order' of a speaker's crossover circuit matters, along with many brands of crossover parts that rob musicality. These are all measurable, but it's usually easier to listen for them via well-conceived experiments.
And there are mechanically-unsound speaker cabinets. To determine for oneself, just buy a $40 stethoscope and have a listen to the sides, back, top... I cannot understand why no reviewer ever bothers with this.
I hope this helps sort some things out, perhaps for future discussions as separate issues, each related to the OP's original well-put question.
Best regards,
Roy Johnson
Green Mountain Audio