I too am going to weigh in on the speaker end, with the provisos that (1) you do have to reach a certain level of amplifier and source performance, which varies from one individual to the next depending on what you can tolerate, and (2) really disparate levels of spending on source/amplification/speakers are bound to lead to a worse result than an intelligent distribution.
Having said that, there is absolutely no question that speakers have by far the biggest deviations and distortions from perfection, that is, they color the original signal far more than any other component, usually by several orders of magnitude! And the worst types of offenses are just "passed over" by the reviewers, e.g., a total lack of time or phase coherence, massive impedance and phase anomalies which put tremendous demands on the amplifier, highly inconsistent diffraction and dispersion patterns, etc. It's as if the reviewers take the attitude that "everyone does it, so it must be okay"! It's not even close to okay, it's just that very few speaker designers have tried to seriously address these issues.
I will make an even bolder statement: If you assembled a system of decent components including a typical 3-way rectangular-box speaker with a typical high-order multiway crossover, and then took a short sample of a complex musical signal and studied it all the way through the chain, it would remain largely intact until it reached the speakers, at which point it would instantly transform into a waveform that is literally nearly unrecognizable compared to the original. Of course, I could be proved wrong, but I doubt it. If anyone has done this, I would love to see a reference to it.
Please do not accuse me of suggesting that measurements are all that matter. I am merely suggesting that screwing up the waveform to such a degree that it hardly bears any resemblance to the original, cannot possibly be a good thing. There is a reason that full-range electrostatics sound so good, and it is that they avoid most of the problems I just mentioned. Of course, they have a whole new set of problems, most of which revolve around unsolvable room interactions, which is why most people don't put up with them.
So while I'm not saying that one should spend more money on speakers than anything else, I am saying that they have the most variability and thus deserve the most effort in searching for one that satisfies.
Having said that, there is absolutely no question that speakers have by far the biggest deviations and distortions from perfection, that is, they color the original signal far more than any other component, usually by several orders of magnitude! And the worst types of offenses are just "passed over" by the reviewers, e.g., a total lack of time or phase coherence, massive impedance and phase anomalies which put tremendous demands on the amplifier, highly inconsistent diffraction and dispersion patterns, etc. It's as if the reviewers take the attitude that "everyone does it, so it must be okay"! It's not even close to okay, it's just that very few speaker designers have tried to seriously address these issues.
I will make an even bolder statement: If you assembled a system of decent components including a typical 3-way rectangular-box speaker with a typical high-order multiway crossover, and then took a short sample of a complex musical signal and studied it all the way through the chain, it would remain largely intact until it reached the speakers, at which point it would instantly transform into a waveform that is literally nearly unrecognizable compared to the original. Of course, I could be proved wrong, but I doubt it. If anyone has done this, I would love to see a reference to it.
Please do not accuse me of suggesting that measurements are all that matter. I am merely suggesting that screwing up the waveform to such a degree that it hardly bears any resemblance to the original, cannot possibly be a good thing. There is a reason that full-range electrostatics sound so good, and it is that they avoid most of the problems I just mentioned. Of course, they have a whole new set of problems, most of which revolve around unsolvable room interactions, which is why most people don't put up with them.
So while I'm not saying that one should spend more money on speakers than anything else, I am saying that they have the most variability and thus deserve the most effort in searching for one that satisfies.