Article: "Do Blind Listening Tests Work? My Sessions with the Colorado Audio Society"


Love this writer. Possibly of interest.

"Many subjective audiophiles loathe blind listening tests. The standard putdown for blind testing is, “That’s not the way I listen.” Yet, in truth, blind comparisons—free from the influence of price, brand, technology, aesthetics, or other personal non-sonic biases—represent the purest form of subjective evaluation. So why aren’t blind tests more popular with audiophiles? The answer is simple—conducting a well-designed, truly unbiased blind test is a pain in the ass. I know, because I just completed one with the help of members of the Colorado Audio Society."

hilde45

@sns 

So the question remains, knowing I'm being influenced by my own biases, why is it important for me to conform to some objective determinant of bias?

Can't quite follow your point, here. Can you restate it for me?

FWIW, I distinguish between a "bias" and a "preference" where a "preference" is something I'm fully aware of and choose, and a "bias" is something that is not completely transparent to me and may be preventing me from fully grasping all that could be at play in my system. A "bias," in my way of contrasting these terms, is an obstacle.

For me. Virtually my entire audiophile life over fifty years was by myself. It’s hard to blindly go up to your system and randomly switch something. Occasionally I would have dealers switch stuff without me knowing… but good analysis requires a couple weeks for each audition not a five minute switch. Not that you can’t differentiate big differences consistently, but meaningful analysis takes time. I guess it would be great if a cable manufacturer would disguise several brands and send them to you to evaluate. 

Good topic.

Aren't we all automatically biased by what our system sounds like when we listen to other systems? Don't we automatically compare the two in our heads? I think enough of us have been to shows and gone into megabuck rooms and were very disappointed by the sound. And vice versa - low buck rooms with toe tapping going on. And if we were gifted either of those systems, we might have enough check marks of sound quality items in our heads to convince ourselves we might be able to push that system to what we really like. And what you like may not be what I like, and that's okay.

I've been to a few club members' systems and liked their gear very much but had a complete list of what improvements "should" be made to really get "better" sound. Great gear is only one of the long list of items that make great sound. Some folks cannot attack all of them and some don't care to.

Some of the blind listening sessions I have been in show me my biases and I'm okay with not being a perfect listening machine. I am as analytical as I can be and am fully guilty of listening to my own system 50% of the time. Last night I threw on some random albums and just listened to tunes. And today I'm gonna go back and repeat some of them because they were enjoyable. 

But from experience, I know that in an unfamiliar room and system where we may be comparing cables or components, I am sure I can hear a difference which may really be a preference (to me). If I can't hear a delta, then, okay, something to noodle on.

Biases are subconscious orientation, tastes,past decisions etc, influencing our perception and evaluation.
 
 They may be positive biases, as a musician asking and waiting for a specific sound in his practice, or negative biases (placebo/nocebo) as an audio fool waiting for a necessary increase in quality because a new purchase...
 
But negative or positive they may be acquired by a long practice history in musicians case, or by sound engineers and acousticians working for decades in their field...Positive biases are expertise...
 
I did not wait to buy a new upgrade to reach a better sound quality...
 
 Happily for budget reason i could not, then i resolved to create my own tweaks and for 2 years, 7 day /7 i experimented with many acoustics concepts in my dedicated room.
 
 (resonators and ASW/LV ratio, absorbing/diffusion/reflective balance, locations of resonators not only their mechanical tuning, non conventional tweaks as Schumann generators grid, ionisation , minerals grounding of the gear pieces, vibrations/resonance controls, speakers and headphones modifications etc )
 
 
 I learned how to acquire new biases created by mastering acoustics concepts.. Which is way better than organizing a complex  double blind test so interesting and instructive it could be for sure as it was for the OP and i dont doubt him on this as a very informative process in itself....
 
 In audio they are two groups:
 
The subjectivist hate blind test.... The objectivist like them...
 
The two groups are driven by a common  gear obsession...One hate blind test because he judge the gear by his hearing biases the other group only look at measuring dial to evaluate the gear, his biases as conflating perceived sounds qualias with Fourier analysis...
 
 But it is the same blindness to the necessary and useful positive acquisition of new biases under experiments driven by  acoustic concepts...
 
One group for example promote cheap low cost dac as good and the costlier one as snake oil in need of blind test...
 
The other group swore by his costlier piece of gear as the best for his "taste"....
 
I grounded my dac (low cost) and put it in acoustical optimal environment.... I used an objective set of conditions regulated by my subjective trained hearing....It was enough for me...
 
I cannot pass a  double blind test because the sound qualities i am used to perceive and evaluate pertain to this specifically designed room of mine, with the piece of gear i specifically modified with tweaks  and others means...It is a unique environment  if we make the sum total of all acoustics factors which exist in only one place : mine...
 
I do not claim to perceive absolute good sound "per se" everywhere in any condition, i claimed to be able to improve only mine with basic concepts...
 
 Double Blind test are industrial tools to test or debunk claims...
 
They are of no use when you test different resonators for example many times each day for 2 years as an acoustic hobby...
 
i learned to trust my ears in these specific conditions...I used simple blind tests hundred of times for 2 years by design and choice tuning resonators or by chance (forgetting to put the Schumann generators button "on" for example )
 
I do not have bat ears no more than a piano tuner had...
 
But i know what is best  in my specific room without need of double blind test which will be impossible to do in my room anyway...
 
Thanks to the OP for this interesting audio society location  and article...
There is even a section dedicated to tweaks...
A great interesting site...
 

Blind testing wouldn't work for me as I don't like listening to music with my eyes closed, anymore. Used to in my younger hazy days..  I have to have a visual placement of instruments to really enjoy it. Like I would never go to a concert and listen with my eyes closed.