How much can be measured -- and how much cannot?

Sabai,

The problem with that is when its all said and done you are still just getting opinions which will lead you to believe that one of three things have occurred; 1) the measurement is inaudible to the most sophisticated ear using the most sophisticated equipment which as to that particular measurement it is meaningless except there it is on the test equipment; 2) that these same folks with the same equipment can hear the the effect of the measurement but thinks it doesn't materially affect the sound they deem important; or 3) that their listening skills or equipment are not up to standard for evaluating equipment, or that they just can't hear it because of their actual hearing limitations. What would a magazine have to gain by pursuing the testing/review you suggest. The finding has to affect the magazine and/or its reviewers negatively.

Interestingly I can think of one internet mag that uses two reviewers on many/all of its review who review the product separately. They publish each review but make no attempt to reconcile any differences which there often are.

Hobby magazines in general rarely publish negative reviews of anything, its just bad for business. As close as they will come is when someone like Adkinson measures spec's and points out deviations from manufacturers spec's or things HE thinks are meaningful for users and lets you draw your own conclusions whether you feel they are relevant, or on a rare occasion a reviewer will parse words in a way that MIGHT alert a potential user that it ain't up to snuff. But the reader has to put on his thinking cap to sort it all out. That is why personal knowledge is so important.

For me, the ears must be the first guide, and are much more important than any spec or measurement. One must listen to many different types of equipment/systems and decide what one's priorities are sonically. Only then can one begin to use measurements and specs to help in a purchasing decision, by learning how and why the equipment types you like sound like they do. At least, this was my approach, since I was/am not very mechanically inclined at all, but I am blessed with very good ears, and as a professional musician I live the "absolute sound" literally almost every day. Very, very often in the world of audio reproduction, the equipment that measures "best" does not actually sound the best. So the actual measurements may not be very important, depending on one's priorities. What is more important, after one has determined one's priorities, is figuring out which measurements/specs are more important to you, and why, based on what you hear/want to hear. Hope this makes sense, I'm tired and probably shouldn't be posting right now. :)

Newbee,
You're absolutely right. You will rarely read a negative review. And if anything negative is said it is usually couched in terms that make it sound not so bad at all, really. Or a matter of personal preference. You really do need to read reviews with your thinking cap on.

Regarding Stereophile reviews, my point is not that we have 2 reviewers commenting on the same aspects of the same products on Stereophile. If this were the case, two differing opinions could easily stand side by side without the need for further explanations.

What we have at Stereophile is 2 reviewers commenting on different aspects of the same products. When you have the latter happening and John Atkinson says "I don't know how he (the reviewer) could have liked the product given my measurements", he is implying that his measurements supersede the ears of his esteemed colleague.

This is a whole other ball game. Under these circumstances, I think it behooves John Atkinson to sit down and have a listen to the component under review with the reviewer whose ears are being called into question. Otherwise readers are left to consider what it all means without any attempt by the magazine to clarify matters.

The choices are:

1. The reviewer must be right since he is the only one of the 2 who actually listened to music on the component.
2. John Atkinson must be right because measurements are more important that the ears of a reviewer.

Leaving readers in limbo to sort out a matter that could have been clarified or reconciled by the 2 people in question getting together (but not doing so) is not the best way to present audio reviews in a widely-read audio magazine for high end consumers. IMO.

LOL, while your choices are correct I do not think they are the only ones available. You might choose to think that not only Atkinson's valuation of the results of the component's deficiencies might be wrong, but so could the reviewers conclusions from his listening sessions.

One of the things all audiophiles experience in evaluating components is recognizing initially everything that is happening at one time. Usually subtle changes brought about by components deficiencies only creeps in with time, sometimes a long time. I'll spare you examples. But in this case I think it sez a lot about JA's integrity (if not his sonic preferences) that after discovering the measurements discrepancy he didn't simply call Fremer and tell him what he measured which would have allowed Fremer to incorporate it in some way in his review. A very pratical solution from a PR point of view - nobody loses and the audience never knows.

Interesting.................

Newbee,
I have no idea what your LOL means. So I will return it in good humor. LOL.

I am assuming the following:

1. That John Atkinson's equipment is working when he takes measurements.
2. That Mr. Fremer and other reviewers are reporting accurately what they hear.

If we cannot assume these 2 basic things then there is no point in this whole exercise because everything becomes smoke and mirrors and nothing can be believed. If you start to question "valuations" of observations of what others report you can turn anything in any direction you wish. IMO.

I believe emphasizing JA's integrity is misplaced here. This is simply the way they do things at Stereophile. Integrity is part of their work, not something that would be extraordinary to expect from them. I am not questioning their integrity at all. I am questioning if there is not something missing in their evaluation process. Something very simple. They finish their work. They read each other's reports. They sit down and listen together and Fremer or another reviewer listens for the measurement side of things and Atkinson listens to the music as well as to any measurement factors he may be able to discern. Simple.

In this way, if one side or the other was missing something they can write a codicil to their report. This means instead of Atkinson saying "I don't understand how the reviewer could like that component after what my measurements show" he might well say after listening to some music that he can actually hear and report some good things -- in spite of what his measurements showed and the valuation he gave to those measurements. And the same for Fremer or another reviewer.

This does not mean Atkinson or a reviewer are changing the valuations placed on measurements or audition of components. Those observations stand. What they are doing is giving a second valuation based on listening in a different way. I mean, this is audio we are talking about, is it not? Or are we talking about the preeminence of the oscilloscope over the ears.