Why blind listening tests are flawed


This may sound like pure flame war bait - but here it is anyway. Since rebuilding my system from scratch, and auditioning everything from preamps to amps to dacs to interconnects to speaker cable etc, it seems clearer than ever.

I notice that I get easily fooled between bad and great sounding gear during blind auditions. Most would say "That should tell you that the quality of the gear is closer than you thought. Trust it".

But it's the process of blind listening tests that's causing the confusion, not a case of what I prefer to believe or justify to myself. And I think I know why it happens.

Understanding the sound of audio gear is process of accumulated memories. You can listen to say new speakers for weeks and love them until you start hearing something that bothers you until you can't stand them anymore.

Subconsciously you're building a library of impressions that continues to fill in the blanks of the overall sound. When all the holes are filled - you finally have a very clear grasp of the sonic signature. But we know that doesn't happen overnight.

This explains why many times you'll love how something sounds until you don't anymore? Anyone experience that? I have - with all 3 B&W speakers upgrades I've made in my life just to name a few.

Swapping out gear short term for blind listening tests is therefore counter productive for accurately understanding the characteristics of any particular piece or system because it causes discontinuity with impression accumulation and becomes subtractive rather than additive. Confusion becomes the guaranteed outcome instead of clarity. In fact it's a systematic unlearning of the sound characteristics as the impression accumulation is randomized. Wish I could think of a simpler way of saying that..

Ok this is getting even further out there but: Also I believe that when you're listening while looking at equipment there are certain anchors that also accumulate. You may hear a high hat that sounds shimmering and subconsciously that impression is associated with some metallic color or other visual aspect of the equipment you happen to be watching or remember.

By looking at (or even mentally picturing) your equipment over time you have an immediate association with its' sound. Sounds strange, but I've noticed this happening myself - and I have no doubt it speeds up the process of getting a peg on the overall sound character.

Obviously blind tests would void that aspect too resulting in less information rather than more for comparison.

Anyone agree with this, because I don't remember hearing this POV before. But I'm sure many others that have stated this because, of course, it happens to be true. ;
larrybou
10-22-14: Atmasphere
The need for blind tests illustrates to me just how little the bench measurements correspond to what we hear. If there was greater correlation, we would not need the blind tests at all.

I was watching one of those detective/forensic shows and they were analyzing the 911 call tape. The Forensic Audio Analyst said, "speech recognition is a science we've put a lot of money into but nothing is as good as the human ear and human perceptual system in understanding speech..." as he sat there in front of his expensive looking equipment and computer monitor displaying waveforms of the call. I'd say the same applies to music.
"Larrybou - what you need to do after listening to cables for days at a time and then declaring a winner is to have someone else switch the losing cable back in without you knowing."

Maybe so. But I wouldn't trust the conclusion until I was able to spend a good number of hours listening to a variety of material on the losing cable. And then if I knew when the cable was switched, and knew it was the losing cable it wouldn't serve any purpose. If I didn't know when the cable was switched, then my memory is being tricked and it would be counter productive since I wouldn't know at what point I was listening to what cable.
"Short-term tests ignore the mechanism of mental schemas, whereby we build mental models of everything we sense. A short test ignores the mind's need to build schemas to understand constructions of various concepts (e.g., sonic signatures, musical values, etc.) and compare their virtues over time."

I knew there was a better way of saying it..
It doesn't matter what a reviewer "thinks", what "religion" he follows...important is, that he writes something which can be sold from a Magazine.
This is the reason that for every customer, buyer, reader is a "special reviewer."
We have one for readers who prefer
- ultra expensive units
- beginner units
- medium priced unit
- technical specs
- analog
- digital
- alternative products from the good old days because they don't trust modern units
- DIY wonders from enthusiastic Amateurs
- blind tests ?
You get what you deserve. What you believe or not, please, no one cares about. Keep the tire rolling, that's all we want :-)
"It doesn't matter what a reviewer "thinks"

Syntax, are you really Dwayne "The Rock" Johnson? :^)