Audio Science Review = "The better the measurement, the better the sound" philosophy

Another story. :)

Our audiophile society was invited to a stereophile reviewer's home to see and experience his setup (he just left that organization).  He had a new amp to review against his own.  Room was too small for all of us to go in there so we divided into two groups.  First group went in and came out.  Without telling us anything, we went in there.  At the end, reviewer asked which amplifier sounded better which folks did.

When we came out to join the larger group, we realized the first group had voted the exact opposite!!!  The reviewer said he had played the amps in reverse order for them vs us.

You see how faulty sighted evaluation is?  

We create controls in listening tests to create reliable outcomes and avoid the above.

Having done various blind tests over the years, it's a very powerful lesson.  It's too bad many audiophiles haven't experienced their 'sighted' impressions dissolving away when they can't use their knowledge of which piece of gear is actually playing.   There's nothing that sinks in like an actual experience.

(And many audiophiles see blind testing as almost synonymous with "detecting no sonic differences."  Where in fact plenty of differences have been detected in blind tests for various things.  That is of course one way various codecs were arrived at.  And I've had some positive results for identifying differences in my own blind tests).

Of all the comments I have read in this thread, this by djones astounded me:

“This is wrong, as a measurement guy "better sound " is subjective and not anything I pay attention to. “

I thought I misinterpreted the comment, but then

No idea if it is better or worse? Seriously? This explains the reply I got on ASR when I asked “What does it sound like”. I was told that doesn’t matter because it measures perfectly. Well let us take one example. I listen to a lot of equipment and have done many comparisons together with friends, at shows and also in demonstrations by companies. My major complaint about a dac that ASR rates as the best measuring dec is that it sounds like crap. I stated that the mid range is shrill, particularly when dealing with female voices. I was categorically told that I was wrong. I am not wrong – for me it sounds shrill. I was thrown off the site for this.

I would not use this dac if it were given to me gratis. Fine, I accept that others may like it, however I do not. Interestingly only one of the 8 that was present at the comparison thought it was satisfactory. We did not know what the brand was at the time. Perhaps the fact that we are all interested in classical music was a deciding factor.

Another example is an amp that ASR wets themselves over as the best measuring amplifier of all time; it is actually quite good, especially for the price, but there are far better amplifiers for my taste.

Regarding Amir’s comments here. Frankly I find them offensive. He has come along with the attitude of "Let me educate you." I do not need him or anyone else to educate me on what I like and dislike. Nor to I need to be lectured that I do not understand the Science. There are many people far more experienced and qualified than Amir of whom I prefer to take note.

@amir_asr

It was a spin and a debating stunt which I called you on.

The part I asked you to respond to was your statement that my "takeaway" after amplifier listening was contrary to audio (and psycho-acoustic) research. The points I re-iterated were quite sound, but I was interested in your counterpoint (as opposed to your talking points).

What you did was elevate the listening test to something it is not, then complain that it doesn’t follow the extensive protocol Dr. Toole used for research. That was improper and I responded to you as such.

Relax mate, I’m not trying to trick you with wicked (sorry, "improper") sophistry. Nor am I running a comprehensive analysis of ASR test methodology, with or without "elevation". No need for all the mansplaining.

The simple point of my earlier post is that we can listen to gear, with some experience and awareness of the pitfalls, without always following the strictest of protocols and still glean meaningful information. I described doing it upthread, you do it yourself with loudspeakers (and you say you think it has value).

Another debating stunt. I do not run a "business" to have a model.

Is there anything to be gained from this semantic quibbling? I’m sure you know that in English "business" has several meanings, including "an activity one is engaged in" and isn’t restricted to commercial activity. If you don’t like the semantics of "business model" just think of it as "modus operandi" or "general approach". You know what I mean. Argue the substance.

And it is not like you have shown any of those editors that hold on to gear perform comparative blind testing of speakers. They have the time according to you but waste it away with who knows what. You want to complain about something, complain about that.

Pure whataboutism. How about those reviewers, eh? Come on.

Perhaps this link is something all people who posted on this site should look at,