Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

There are actually more facets to the Pepsi story. Look up the "Pepsi Challenge" on Wikipedia for several additional bits of information.

The reason Pepsi started doing a single-blind Pepsi vs. Coke challenge decades ago was because Pepsi was often preferred over Coke if the test was based on a single sip of cola. (Most people think this is due to the fact that Pepsi tastes a little sweeter than Coke.) However, Coke often was preferred when consumption occurred over a more extended period of time. This sounds a bit like nonoise's ideas about extended listening vs. short-term listening.

If you're really into these issues, take a look at a great study of how brain damage to the prefrontal cortex can affect blinded or semi-blind taste testing: 

 

This study showed that taste preferences were influenced by knowledge of the brand of cola, except in people with damage to the prefrontal cortex of the brain. I don't know if this study has any implications for drinking wine, but it sounds like it could.  

 

Interesting Pepsi Story! Thanks!

 

 

Pepsi is more " biting" then seems more interesting in the short term for me, but if i would drink cola long term each day i will probably set for Coke which is more body friendly for me or less agressive and less "surprizing" in a wrong way ... 😁😊

In the long run even wine cannot beat water source...No water taste the same by the way...

Wine is the same than acoustic it must be learned...Like sex which is not only complex gymnastic anyway but a loving and educated taste experience so to speak......

It remind me of sex in the long run with someone you love compared to a short run with a beautiful unknown woman you dont fall in love with and either she .... this short relation may act like a drug rush thats all...If you ever love a woman you know why...And if you are loved especially ...

There is a part of taste, sound and sexe experience that dont work the same either long run and short run in term of evaluation and experience.....

You cannot taste and evaluate in the short run and in the big run some bread like a very big load of sweets so good they are... One is for each day a necessary addition the other not...Their action on the body is not the same...One replenish the body the other act more like a drug...

You cannot taste and tune and evaluate rightfully the ratio LEV/ASW , nor timbre perception in the short run....And it is more easy to get SOME dynamic improved and SOME bass a bit better in the short run it work like a load of sweets like a drug...It is the reason why most people at best stop acoustic experiments after buying some costly or cheap acoustic panels...

And if we listen music an hour a day. the level of sound may be liked VERY HIGH it act like a drug on the body ... But if we listen 5 hours of music each day , the sound levels will be prefered lower...

And sorry but nevermind the taste, heavy rock dont work like a string quintet at all on the mind and on the body in the short term and in the long run either...

 

And everybody know that the evaluation of music and sound will be influenced by the brand name visibility...

It is one of the reason why my acoustic listening experiments were so enlightening for me... i became conscious how secondary are the brand name of gear piece for acoustic science results...I supposed here not too much difference in design qualities for sure between gear choices on the S.Q and design quality and price ratio scale... A 100 dollars amplifier is not a 50,000 bucks one...

Acoustic can make miracles change yes, but dont transform stone in bread like Christ did once ...

Any basically relatively good piece of gear will do the job, but not any room will help the system and would do his job to transform it on another level of experience... We need acoustic basic knowledge for that...

Acoustic is more important than the gear choice, especially if neither of your possible choices are completely wrong for sure or if one is not at the high end level and the other at the lowest level for sur.

..Audio magazine are like Colas publicity nothing more...Acoustic and psycho-acoustic rule with ,mechanical vibrations control and electrical noise floor control all audio experience........

Buying and plugging in the wall is not enough sorry at any price ...

 

My 500 bucks audio system  teach me so  with murmur to my ears !

 

😁😊

 

There are actually more facets to the Pepsi story. Look up the "Pepsi Challenge" on Wikipedia for several additional bits of information.

The reason Pepsi started doing a single-blind Pepsi vs. Coke challenge decades ago was because Pepsi was often preferred over Coke if the test was based on a single sip of cola. (Most people think this is due to the fact that Pepsi tastes a little sweeter than Coke.) However, Coke often was preferred when consumption occurred over a more extended period of time. This sounds a bit like nonoise’s ideas about extended listening vs. short-term listening.

If you’re really into these issues, take a look at a great study of how brain damage to the prefrontal cortex can affect blinded or semi-blind taste testing:

https://academic.oup.com/scan/article/3/1/1/1609184

 

This study showed that taste preferences were influenced by knowledge of the brand of cola, except in people with damage to the prefrontal cortex of the brain. I don’t know if this study has any implications for drinking wine, but it sounds like it could.

 

Again there appears to be a strange belief that expectation bias only works one way.  I myself have been surprised on occasions where I expected a component to be bad, based upon price, reviews or friends comments, only to find the item did in fact exceed my expectations.

 

I just did a blind test between my Benchmark LA4 preamp and my Conrad Johnson premier 16LS2 tubed preamp.

Blind testing can be a hassle so I indulge in it fairly rarely, but I've done a few over the years.  Fortunately the Benchmark preamp made a blind test pretty easy in terms of matching levels and quick switching (with the help of my son - we randomized switching etc).

Results of the first test were an easy 15/15 correct guesses as to which preamp was playing.   I'm still a fan of toobs!