How much can be measured -- and how much cannot?


There has been a lot of discussion over the years on Audiogon regarding the measurement of components and other audio products. Some people claim everything is either measurable now or will one day become measurable with more sophisticated measuring equipment. But others say there are things in high end audio that will never me measurable and that measurements are really not that important.

Here is a typical example -- a quote taken from the Stereophile forum regarding their review of the Playback Designs MPS-5:

"JA 2/17/10 Review Measurements of Playback Designs MPS-5
Posted: April 13, 2011 - 8:42am

John Atkinson's 2/17/10 review measurements of the Playback Designs MPS-5 revealed less than stellar technical performance even though Michael Fremer really liked the player. I've included JA's closing measurement remarks below followed by the manufacturer's comments.

To my knowledge there was never any followup in Stereophile regarding the manufacturers reply the MPS-5 could not be adequately measured with traditional measurement techniques.

I believe Stereophile should respond to this reply in the interests of its own measurements credibility.

Len"

How important do you think measurements are? Are the ears really the only true arbiter?
sabai
Unsound,
You say "Things like frequency response and dBW just a couple of examples of measurements that can be important ...". I agree. There are specific measurements that can give us useful information and can help eliminate contenders when there are clear compatibility issues.

But my point is that once this process has taken place and there are a number of possible contenders left in the ring, all with compatible measurements, you can never tell which one will be the "best" for your system until you let your ears take the reigns. IMO. Measurements can never tell us everything we need to know. Once obvious mismatches are eliminated, measurements may tell us nothing further about the sound. The next step is system synergy which is totally unpredictable. IMO. This is where the ears are the final arbiter.

Charles1dad,
You hit the nail on the head. What do John Atkinson's measurements really tell us about the listening experience if the reviewer is in complete contradiction with his measurements and accompanying observations? Not much. IMO.

Onhwy61,
When I talk about the "essence of music" I mean the listening experience itself -- outside of intellectual considerations that consist of various observations that we make about the listening experience, sometimes kept to ourselves, sometimes shared with others.
Unsound, I have certainly heard the impact of some of these, but I don't know of much concern with any of these save what is audible. A tuner or an amp that fails to deal with such issues probably will not sell given how it performs or sounds.
Newbee,
You make a heck of a lot of good points. Don't you think it would be interesting to hear the cross-talk between Fremer and other reviewers and John Atkinson after both have finished their written observations if they sat down together and listened to the equipment being reviewed? It seems to me to be a very logical next step for Stereophile to take with their reviews. A no-brainer, really. I wonder why this is not being done by them?
Newbee and Unsound, I certainly would take no exception to your centering on specs that you find closely associated to your tastes, but I will continue to trust my ears. I have never found any measures other than whether the unit is on or not that are associated with quality music reproduction. Michael Fremer and I certainly are in total agreement about the WAVAC SH-833 monoblock amps. He is the only reviewer in Stereophile or TAS, that I would trust in recommending a component.

At one RMAF, John Atkinson presented a seminar where he had a Boulder amp and an unnamed amp hooked to a THD meter. Under various loads the unnamed amp was horrible and the Boulder exemplary. He went on and on. He had no capability to listen to the two amps. Having heard the Boulder and not liking it, I asked whether he thought the designer of the unnamed amp thought THD was a major concern in design. He was flabbergasted as were most in the audience. I left.
Sabai,

The problem with that is when its all said and done you are still just getting opinions which will lead you to believe that one of three things have occurred; 1) the measurement is inaudible to the most sophisticated ear using the most sophisticated equipment which as to that particular measurement it is meaningless except there it is on the test equipment; 2) that these same folks with the same equipment can hear the the effect of the measurement but thinks it doesn't materially affect the sound they deem important; or 3) that their listening skills or equipment are not up to standard for evaluating equipment, or that they just can't hear it because of their actual hearing limitations. What would a magazine have to gain by pursuing the testing/review you suggest. The finding has to affect the magazine and/or its reviewers negatively.

Interestingly I can think of one internet mag that uses two reviewers on many/all of its review who review the product separately. They publish each review but make no attempt to reconcile any differences which there often are.

Hobby magazines in general rarely publish negative reviews of anything, its just bad for business. As close as they will come is when someone like Adkinson measures spec's and points out deviations from manufacturers spec's or things HE thinks are meaningful for users and lets you draw your own conclusions whether you feel they are relevant, or on a rare occasion a reviewer will parse words in a way that MIGHT alert a potential user that it ain't up to snuff. But the reader has to put on his thinking cap to sort it all out. That is why personal knowledge is so important.