Stereophile has technical measurements to go along with their subjective reviews... they've had it for years. And it's interesting that the reviewer's subjective views don't always align well with the objective measurements.
Let's face it, there are just too many variables. The component under review is part of a system where many complex interactions take place. When the component is sent for measurements it is not part of the same system (being measured on the bench with the technician's test equipment).
Regardless of that, if you know how to interpret the data it will provide some useful insight into the performance of the component under scrutiny.
As an audio reviewer, my particular publication does not have the facility to provide test data. So to double-check my own impressions I have two different reference systems in my home. That way I can try the review gear in both systems (with different associated gear) and see what characteristics remain constant... and also see if any outliers pop up. Often times I find that many of the sonic characteristics of the component remain constant from one system to the other... though there have been exceptions.
For whatever reason I find that system cables are the most unpredictable of my components when going from one system to the other and for me are more difficult to pin down and review than the basic hardware like amps, preamps, and DACs.
Still, I know I can depend on certain cables to do specific things and I can use them to change the tonality of a system more to my liking... kind of like an equalizer. And I'm aware that many audiophiles view system cables as "tone controls."
The simple answer is that there are entirely too many variables to find a system of quality assessment that would satisfy every listener. In my own small group of audio friends their taste in sound is all over the map and runs the gamut from the accurate and precise to the mellow and ill-focused. It seems that a lot of folks focus on different areas of performance that are important to them at the expense of other parameters.
I personally strive for neutral and natural and try to get the most detail possible without losing sight of musicality. And even this description will mean different things to different people.
Also, if you had a system that measured flat +/- 0.5dB 20Hz to 20kHz it would sound unbearably bright and unlistenable to the vast majority of audiophiles and music lovers. So maybe the thing to pursue is which curve or amount of high frequency roll off starting at what frequency would provide the ideal musical sonic balance to the highest percentage of listeners??? We can save matters of dynamic compression, transient speed, and smoothness and transparency for another time.
Frank A
