How much can be measured -- and how much cannot?


There has been a lot of discussion over the years on Audiogon regarding the measurement of components and other audio products. Some people claim everything is either measurable now or will one day become measurable with more sophisticated measuring equipment. But others say there are things in high end audio that will never me measurable and that measurements are really not that important.

Here is a typical example -- a quote taken from the Stereophile forum regarding their review of the Playback Designs MPS-5:

"JA 2/17/10 Review Measurements of Playback Designs MPS-5
Posted: April 13, 2011 - 8:42am

John Atkinson's 2/17/10 review measurements of the Playback Designs MPS-5 revealed less than stellar technical performance even though Michael Fremer really liked the player. I've included JA's closing measurement remarks below followed by the manufacturer's comments.

To my knowledge there was never any followup in Stereophile regarding the manufacturers reply the MPS-5 could not be adequately measured with traditional measurement techniques.

I believe Stereophile should respond to this reply in the interests of its own measurements credibility.

Len"

How important do you think measurements are? Are the ears really the only true arbiter?
sabai
Newbee,
Thanks for explaining. No problem.

I agree with you about not having enough experience. Years ago I was in the very same position you found yourself in -- not enough experience and lots of information trying to influence my purchases coming from various vested interests. This is a very steep learning curve that really never ends.

This has been a very rewarding if expensive process that is nearing an end because of serious financial constraints. But not only because of the latter. My system has got to such a satisfying level that I am at the point of diminishing returns. It would take a lot of money to get to a much higher level and I don't know if I did get there that all the effort and expense would translate into much more listener enjoyment because of the high level my system has reached.

It's time to sit back and enjoy what I have without chasing the latest and omnipresent breakthrough products with the attendant manufacturer claims. Over the years most of the improvements to my system have been what I would term incremental. Certainly, there have been some increments that have been more impressive than others. Some I would even call game-changers in their class. Nevertheless the effect on the sound has been incremental and improvement has not always come with the addition of an expensive component or cable. Sometimes even a very inexpensive tweak has given a wonderful improvement in the sound.

In spite of the impressive claims of even the most respected manufacturers in favor of a single component or cable being able to transform one's system, there has not been a single purchase that comes to mind that has measured up to the inflated claims of the maker.
Unsound,
The technical points you make are excellent and clearly explained. Regarding "audio rags", I would have thought it would be in their best interest to be as clear as possible although reading some of their reviews leads me to believe they seem to think otherwise. I will not name names but I think you will be able to easily find fine examples of unintelligible use of the English language by perusing some of the major audio review magazines. They often feature the most convoluted and confusing language that I can imagine a writer composing about audio matters. Florid language and audio cliches abound.

For the life of me, I cannot see how contradictory or confusing reports help the interests of audio magazines. They only lead to skepticism on the part of savvy readers, IMO. Each side of the coin in Stereophile reviews is valid. But if the two sides contradict each other and the review leaves it all hanging in mid-air it begs the question: don't these fellows realize their business is unfinished?

An example of this was the Stereophile review of Playback Designs MPS-5. Let me preface my remarks by saying I have no affiliation with Playback, I have never had any contact with them, I don't own their player, I have never heard it and I am completely neutral regarding it. I am not in the audio business and I have no connection with any audio company.

John Atkinson states at the end of his measurements: "So while I was impressed by the player's standard of construction, I can't say the same about its technical performance. The relatively high level of background noise limits the MPS-5's resolution with SACD and external 24-bit data to not much better than 16-bit CD. I am puzzled, therefore, why Michael Fremer liked the sound of this player so much."

Could John Atkinson not have picked up the phone and given Michael Fremer a call to arrange a chat and an audition together of the component in question? I mean, this seems to me to be a no-brainer. They work for the same publication. Are they not on speaking terms? You have one voice pointing north and the other voice pointing south and a little voice is heard in the background saying to their readers "up to you". How have they served their readers other than going to the trouble to confuse them? Frankly, I find this a very curious way for John Atkinson to end a review of one of the major contenders in the high end CD player market.
Unsound, when measurements were added to reviews I applauded it. However, I have seen little correlation between them. Nor have I heard good measuring components to sound better nor good reviewing components to sound better. Related to this is double blind testing. Having participated in several of these and having read critiques of the use of 30 sec. same/different tests in psychology, I find them invalid.

I really think that components should be sent to a known room where we have made a recording and had the performance digitized. Then we compare that component inserted into a known system whose measurements are known. We would then compare the change noted from both the existing system and the actual recording. The closer to the initial recording the better.

Screw all other measurements.
TBG, you've already posted that you've found some speaker measurements helpful. I can't help but wonder if upon further examination, you might find some correlation between some other measurements and some sound characteristics.
I like your proposal, very much!
Again, I think dismissing all other measurements would be imprudent.
Unsound, I can hear frequency response and sense dispersion also. I also think I can hear phase problems in speakers. I cannot recall ever getting such measurements before deciding on a speaker. But I cannot really understand a speaker designer who wouldn't use such measurements. I must say that I have sought and appreciate single driver speaker systems and two way systems.