If I may weigh-in here, I have the notion, that a good part of the reason for the measurement aspect of stereo gear is the fact that all components are designed to be interchangeable. Mix and match, if you will. Since the other gear that may be used with any one component is not known my the mfr., an attempt is made to have a "benchmark". This "benchmark" has resulted in a relatively arbitrary, but well meaning, set of measured standards that are meant to assure the user of conformity and usefulness with other units measured by the same standards. So, standards of measurement were adopted for various aspects of performance such as frequency response, total harmonic distortion, power output, etc.
The problem with this is that these processes assume a given idea that flat frequency response or low THD will allow accurate pass-thru to the next component with flat frequency response or low THD. While this may superficially seem to be the case, it is not. Little or no regard is given to the additive or interactive effects of the other components in the chain because the designer is unaware of what the other components will be. So this leads to the designing of equipment in a "vacuum", so to speak. Only the most basic "standards" like output level and impedance levels under static conditions are even considered. All of the other issues are left to the consumer to determine, regarding which items may work well together. And the consumer is ill-equipped to make these decisions, because there is little, if any, data provided for this, and most consumers would not know how to use such data anyway.
This is what leads to the mysterious "synergy" discussions, and the apparent "disconnect" between measurements and sound quality. It is not that the measurements are bad, it is the basic idea of what should be measured, and how to measure it,and provide useful data, that is at the root of the problem.
If a consumer knew that his amp exhibited a THD profile of primarily even-ordered harmonic distortion with a major part at the 2nd harmonic, he could choose a speaker that also had a major part of its THD in the 2nd harmonic,and wire them 180 degrees out of phase, thereby cancelling a significant portion of the distortions of the amp/speaker combination. But, nobody seems to be aware of this type of "system integration design" and none of the measurements really are geared to help anyone do this. Everybody just wants flat response and low distortion, but all components have some dips or rises, and all have some components of distortion. Failure to correctly match these, and other, characteristics will result in additive distortions or frequency anomalies. The consumer is frequently unaware that he is even making this mistake. Also, there is a big difference in having two-tenths of a percent of distortion at the 2nd harmonic, and having one-tenth of a percent at every harmonic point up the scale. In the first case there is a relatively small amount of distortion at one point on the curve. In the second case, there is a smaller maximum distortion rating, but the distortion is all over the place.
So, to sum up, I do believe, as Paulwp seems to, that things that are heard can be measured. But the things measured, and the ways they are measured, and the applications of those measurements leave much to be desired. Now, add that to the things that we haven't learned how to measure, or even realize that they need to be measured, and we end up with a reliance on a faulty set of measurements, and misunderstanding of same, that cannot accurately be used to select our equipment. Thus, many of us rely on our ears to measure what we do know, the sound.
Ultimately the ear is the judge. But I do not discount that certain measurements can lead one to an informed platform from which to begin auditioning, if one can adequately interpret the data that can be found, and apply it in a meaningful way, resulting in a happy combination of components(synergy). And although I personally am a proponent of listening as the final arbiter, I do use design data and measurement data when I can find ways to apply it.
Regarding the brain's interpretation of the sounds generated by the system, I am not in an informed position to comment on that. But I do find it interesting.