2. tech-talk
  3. 1332143340

How much can be measured -- and how much cannot?

There has been a lot of discussion over the years on Audiogon regarding the measurement of components and other audio products. Some people claim everything is either measurable now or will one day become measurable with more sophisticated measuring equipment. But others say there are things in high end audio that will never me measurable and that measurements are really not that important.

Here is a typical example -- a quote taken from the Stereophile forum regarding their review of the Playback Designs MPS-5:

"JA 2/17/10 Review Measurements of Playback Designs MPS-5
Posted: April 13, 2011 - 8:42am

John Atkinson's 2/17/10 review measurements of the Playback Designs MPS-5 revealed less than stellar technical performance even though Michael Fremer really liked the player. I've included JA's closing measurement remarks below followed by the manufacturer's comments.

To my knowledge there was never any followup in Stereophile regarding the manufacturers reply the MPS-5 could not be adequately measured with traditional measurement techniques.

I believe Stereophile should respond to this reply in the interests of its own measurements credibility.

Len"

How important do you think measurements are? Are the ears really the only true arbiter?
sabai
sabai OP
2,027 posts
 
Newbee,
You're absolutely right. You will rarely read a negative review. And if anything negative is said it is usually couched in terms that make it sound not so bad at all, really. Or a matter of personal preference. You really do need to read reviews with your thinking cap on.

Regarding Stereophile reviews, my point is not that we have 2 reviewers commenting on the same aspects of the same products on Stereophile. If this were the case, two differing opinions could easily stand side by side without the need for further explanations.

What we have at Stereophile is 2 reviewers commenting on different aspects of the same products. When you have the latter happening and John Atkinson says "I don't know how he (the reviewer) could have liked the product given my measurements", he is implying that his measurements supersede the ears of his esteemed colleague.

This is a whole other ball game. Under these circumstances, I think it behooves John Atkinson to sit down and have a listen to the component under review with the reviewer whose ears are being called into question. Otherwise readers are left to consider what it all means without any attempt by the magazine to clarify matters.

The choices are:

1. The reviewer must be right since he is the only one of the 2 who actually listened to music on the component.
2. John Atkinson must be right because measurements are more important that the ears of a reviewer.

Leaving readers in limbo to sort out a matter that could have been clarified or reconciled by the 2 people in question getting together (but not doing so) is not the best way to present audio reviews in a widely-read audio magazine for high end consumers. IMO.
newbee
4,792 posts
 
LOL, while your choices are correct I do not think they are the only ones available. You might choose to think that not only Atkinson's valuation of the results of the component's deficiencies might be wrong, but so could the reviewers conclusions from his listening sessions.

One of the things all audiophiles experience in evaluating components is recognizing initially everything that is happening at one time. Usually subtle changes brought about by components deficiencies only creeps in with time, sometimes a long time. I'll spare you examples. But in this case I think it sez a lot about JA's integrity (if not his sonic preferences) that after discovering the measurements discrepancy he didn't simply call Fremer and tell him what he measured which would have allowed Fremer to incorporate it in some way in his review. A very pratical solution from a PR point of view - nobody loses and the audience never knows.

Interesting.................
sabai OP
2,027 posts
 
Newbee,
I have no idea what your LOL means. So I will return it in good humor. LOL.

I am assuming the following:

1. That John Atkinson's equipment is working when he takes measurements.
2. That Mr. Fremer and other reviewers are reporting accurately what they hear.

If we cannot assume these 2 basic things then there is no point in this whole exercise because everything becomes smoke and mirrors and nothing can be believed. If you start to question "valuations" of observations of what others report you can turn anything in any direction you wish. IMO.

I believe emphasizing JA's integrity is misplaced here. This is simply the way they do things at Stereophile. Integrity is part of their work, not something that would be extraordinary to expect from them. I am not questioning their integrity at all. I am questioning if there is not something missing in their evaluation process. Something very simple. They finish their work. They read each other's reports. They sit down and listen together and Fremer or another reviewer listens for the measurement side of things and Atkinson listens to the music as well as to any measurement factors he may be able to discern. Simple.

In this way, if one side or the other was missing something they can write a codicil to their report. This means instead of Atkinson saying "I don't understand how the reviewer could like that component after what my measurements show" he might well say after listening to some music that he can actually hear and report some good things -- in spite of what his measurements showed and the valuation he gave to those measurements. And the same for Fremer or another reviewer.

This does not mean Atkinson or a reviewer are changing the valuations placed on measurements or audition of components. Those observations stand. What they are doing is giving a second valuation based on listening in a different way. I mean, this is audio we are talking about, is it not? Or are we talking about the preeminence of the oscilloscope over the ears.
tbg
5,263 posts
 
Newbee, you are right that listening entails tinkering with speaker placement, compatible electronics, etc. and the tastes of the reviewer. While measurements are more stable, but also they ignore the subtleties of tweaking with placement, etc. But obviously many of us with some experience with measurement perfect electronics, know that listening by the customer is the only real guide.
newbee
4,792 posts
 
Sabai, More than anything else I was laughing at my self for allowing myself to be drawn into a discussion of magazine reviews/reviewers. I truly am an agnostic when it comes to most anything they publish. I haven't subscribed to any in years. That said I do believe JA's measurements can be helpful to those who know how to use them and I thought I should defend them on that issue alone.

In the 'beginning' I wasted a lot of money primarily based on published reports,tests,opinions, and a certain salesmanship pushing me on to chasing the latest, best etc. I waited anxiously for the next issue to arrive each month. Frankly, I had insufficient experience to make judgments on my own and they were my primary source.

Then I found this wonderful source of information - The Internet. Wow, now I had at my disposal user's as well as designer's and manufacturer's comments on all things audio! A treasure trove of information which I used as a basis for forward progress. It was broad band info though and part of the exercise was discarding all of the BS from salesmen and proud tyros. But it was there for those who are interested.

My LOL was not about you. Forgive me if my construction implied otherwise.
More to discover