Article: "Do Blind Listening Tests Work? My Sessions with the Colorado Audio Society"


Love this writer. Possibly of interest.

"Many subjective audiophiles loathe blind listening tests. The standard putdown for blind testing is, “That’s not the way I listen.” Yet, in truth, blind comparisons—free from the influence of price, brand, technology, aesthetics, or other personal non-sonic biases—represent the purest form of subjective evaluation. So why aren’t blind tests more popular with audiophiles? The answer is simple—conducting a well-designed, truly unbiased blind test is a pain in the ass. I know, because I just completed one with the help of members of the Colorado Audio Society."

hilde45
Biases are subconscious orientation, tastes,past decisions etc, influencing our perception and evaluation.
 
 They may be positive biases, as a musician asking and waiting for a specific sound in his practice, or negative biases (placebo/nocebo) as an audio fool waiting for a necessary increase in quality because a new purchase...
 
But negative or positive they may be acquired by a long practice history in musicians case, or by sound engineers and acousticians working for decades in their field...Positive biases are expertise...
 
I did not wait to buy a new upgrade to reach a better sound quality...
 
 Happily for budget reason i could not, then i resolved to create my own tweaks and for 2 years, 7 day /7 i experimented with many acoustics concepts in my dedicated room.
 
 (resonators and ASW/LV ratio, absorbing/diffusion/reflective balance, locations of resonators not only their mechanical tuning, non conventional tweaks as Schumann generators grid, ionisation , minerals grounding of the gear pieces, vibrations/resonance controls, speakers and headphones modifications etc )
 
 
 I learned how to acquire new biases created by mastering acoustics concepts.. Which is way better than organizing a complex  double blind test so interesting and instructive it could be for sure as it was for the OP and i dont doubt him on this as a very informative process in itself....
 
 In audio they are two groups:
 
The subjectivist hate blind test.... The objectivist like them...
 
The two groups are driven by a common  gear obsession...One hate blind test because he judge the gear by his hearing biases the other group only look at measuring dial to evaluate the gear, his biases as conflating perceived sounds qualias with Fourier analysis...
 
 But it is the same blindness to the necessary and useful positive acquisition of new biases under experiments driven by  acoustic concepts...
 
One group for example promote cheap low cost dac as good and the costlier one as snake oil in need of blind test...
 
The other group swore by his costlier piece of gear as the best for his "taste"....
 
I grounded my dac (low cost) and put it in acoustical optimal environment.... I used an objective set of conditions regulated by my subjective trained hearing....It was enough for me...
 
I cannot pass a  double blind test because the sound qualities i am used to perceive and evaluate pertain to this specifically designed room of mine, with the piece of gear i specifically modified with tweaks  and others means...It is a unique environment  if we make the sum total of all acoustics factors which exist in only one place : mine...
 
I do not claim to perceive absolute good sound "per se" everywhere in any condition, i claimed to be able to improve only mine with basic concepts...
 
 Double Blind test are industrial tools to test or debunk claims...
 
They are of no use when you test different resonators for example many times each day for 2 years as an acoustic hobby...
 
i learned to trust my ears in these specific conditions...I used simple blind tests hundred of times for 2 years by design and choice tuning resonators or by chance (forgetting to put the Schumann generators button "on" for example )
 
I do not have bat ears no more than a piano tuner had...
 
But i know what is best  in my specific room without need of double blind test which will be impossible to do in my room anyway...
 
Thanks to the OP for this interesting audio society location  and article...
There is even a section dedicated to tweaks...
A great interesting site...
 

Blind testing wouldn't work for me as I don't like listening to music with my eyes closed, anymore. Used to in my younger hazy days..  I have to have a visual placement of instruments to really enjoy it. Like I would never go to a concert and listen with my eyes closed.

Interesting. Since my system/room improvement days i observed i prefered to listen music eyes open...

It is especially true with my headphone system... I like to see the music in front of me in my room ..

 

Blind testing wouldn't work for me as I don't like listening to music with my eyes closed, anymore. Used to in my younger hazy days..  I have to have a visual placement of instruments to really enjoy it. Like I would never go to a concert and listen with my eyes closed.

 

One factor affecting the results that was not accounted for in the test is the listening skills of the participants.  I have read, but I don't have a link, that when people receive some training in critical listening, they do better at distinguishing between different components in these kinds of tests.

@tomcy6

+1 Listening skills... absolutely. I occasionally get together with some folks that have excellent listening skills. There is no bias and preferences in the observations and conclusions. We exchange critical analysis of the sound... then add on whether we like it or not. 

Many folks seem to think that everyone is picking their preferences and not analyzing sound qualities. All of the audiophiles I know can easily discern the differences between personal preferences and sonic characteristics. Granted, beginners probably have a hard time unraveling personal preferences from sonic characteristics.