Why Do So Many Audiophiles Reject Blind Testing Of Audio Components?


Because it was scientifically proven to be useless more than 60 years ago.

A speech scientist by the name of Irwin Pollack have conducted an experiment in the early 1950s. In a blind ABX listening test, he asked people to distinguish minimal pairs of consonants (like “r” and “l”, or “t” and “p”).

He found out that listeners had no problem telling these consonants apart when they were played back immediately one after the other. But as he increased the pause between the playbacks, the listener’s ability to distinguish between them diminished. Once the time separating the sounds exceeded 10-15 milliseconds (approximately 1/100th of a second), people had a really hard time telling obviously different sounds apart. Their answers became statistically no better than a random guess.

If you are interested in the science of these things, here’s a nice summary:

Categorical and noncategorical modes of speech perception along the voicing continuum

Since then, the experiment was repeated many times (last major update in 2000, Reliability of a dichotic consonant-vowel pairs task using an ABX procedure.)

So reliably recognizing the difference between similar sounds in an ABX environment is impossible. 15ms playback gap, and the listener’s guess becomes no better than random. This happens because humans don't have any meaningful waveform memory. We cannot exactly recall the sound itself, and rely on various mental models for comparison. It takes time and effort to develop these models, thus making us really bad at playing "spot the sonic difference right now and here" game.

Also, please note that the experimenters were using the sounds of speech. Human ears have significantly better resolution and discrimination in the speech spectrum. If a comparison method is not working well with speech, it would not work at all with music.

So the “double blind testing” crowd is worshiping an ABX protocol that was scientifically proven more than 60 years ago to be completely unsuitable for telling similar sounds apart. And they insist all the other methods are “unscientific.”

The irony seems to be lost on them.

Why do so many audiophiles reject blind testing of audio components? - Quora
128x128artemus_5

@big_greg

"When you meet a new person can you look at them for a few seconds and determine how they compare to the last person you met, or does it take time for their true nature and character to reveal itself?

For me, it’s the same with audio components.

It takes time to get to know them."

 

Fair enough, if you believe there might be parallels between comparing a human being with an electronic device - I can't fathom certain human beings even after knowing them for decades but that's another story - but what about those who went around espousing "night and day" differences, yet dare not risk undertaking a blind listening test between a $10 DAC and one costing 100 times as much?

Let’s not forget that human perception evolved primarily to detect differences, which often meant danger. It’s something we’re very good at.

Therefore if it really does take several days/ weeks to identify a sonic difference, which might only be due to a subtle anomaly in frequency response, then just how important could it be anyway?

Furthermore, wouldn’t such delicate differences between frequency response will inevitably suit some material/systems and not others?

In fact you could argue that when the detection of extremely marginal differences (which may well be down to manufacturing tolerances) takes such a long period of time how can we be sure that it’s not our mood/attitude that is actually changing instead of any increase in our perception?

You can bet that the equipment on test will always be slitkre consistent than the human being doing the testing by ears alone.

It’s quite one thing to say I love product X, it’s better built, has better backup service, but entirely another to say it’s clearly sonically better than product Y.

Sonic differences are, I suspect, easily the most important factor when it comes to purchasing new equipment. Audiophiles change equipment in an attempt to upgrade their sound.

I can’t see any reason why us consumers would have a problem with blind listening tests when auditioning potential upgrades. Money and sonic satisfaction are very important to most of us, aren’t they?

So perhaps we should also ask that why is it primarily manufacturers, dealers and reviewers that have an issue with blind listening tests? Instead of embracing an additional way of evaluation, some them seem to be quite hostile.

 

Reviewers/shills/ad men/hacks/sales reps etc seem to be particularly cowardly/defensive/guarded/silent when it comes to reviewing blind.

In fact I know not of a single one that would risk their ’reputation’ in this way.

Not one.

Why is this?

Perhaps their fear of destroying their entire retail business model by revealing the emperor’s new clothes syndrome is real enough.

It should be easy enough to arrange for most, but can you imagine any dealer offering the facilities for prospective customers to listen blind?

Me neither.

@cd318 Thank you for reinforcing my point.  The differences are often subtle and it takes me extended listening to discover those differences and see how I react to them. 

I have nothing "against" blind testing.  Blind test your heart out.  I don't however think it's an effective way for me to judge the sound quality of audio equipment. 

The thing with extended auditioning is that there is a lot of adaptation and not working using short term memory. 

Kind of like going to the eye doctor, the changes in the lenses magnification is deliberately quick because one's brain is working in short term memory and very good at knowing what is good for him. Eye doctors don't send you with a lens to test for a week and see how your eye feel. 

There are a lot of papers discussing this subject in the Audio Engineering Society. I wish I could post some parts but my petition was denied. If one is really interested in these subjects it is worth joining, but be prepared to read long papers with a lot of math and statistics.

Fun story, a few years back, I had my eye doctor and his dad, also an eye doctor, and I asked the young one if he knew why they change the lenses so fast and he said because he usually want to finish quickly, his father almost gave him a spank and asked if I knew or I was being me curious so he told me that  short memory  key in finding the right lenses, because the short memory comparison is key. 

Many times with my dealer we discussed why he plays one short passage and switch, and he says that after their blind test you should have 2 components you like over  the others, and if you are unsure then buy the cheapest.

My dealer also referrers to the long auditioning as a good way to find those things that bother one, like interfaces, switches, esthetics, and anything that is in the functionality domain and see if you like the sound in your home.

All the times I leverage the long term, I only bought something else because something bothered me, I am autistic and some things just don't fit in my brain. Even if the component sounds better, whatever it is distract me from listening. 

English as nth language.

That is why I only like to make changes that are immediately noticeable and not subtle. The subtle ones I am sure add up over the long term, it is just tougher to appreciate and justify them.

@astolfor

"...his father almost gave him a spank and asked if I knew or I was being me curious so he told me that short memory key in finding the right lenses, because the short memory comparison is key"

 

Thanks for sharing that interesting post.

Short term memory comparison does seem to make a lot of sense whenever we have to rely upon our senses. Everytime I look at a picture or a scene and then close my eyes to visualise it, I’m always a little disappointed to find upon opening my eyes once more just how much detail and nuance had faded so fast.

As for autism, unfortunately it still seems to be a mysterious condition, even in 2022.

Best of luck to you.