Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

@chayro You raise a good point. IMHE, the effect is magnified swapping components after just a track or two. I've seen this many times at local audio club and store demo events. 

When I do store demos, I find it much more helpful to listen for an hour or two, switch and replay the same tracks on the 2nd component. Sure, going weeks or more would be better, but excluding home demos, that's just not happening. Unfortunately, break-in issues make "30day return" home demos on new gear hard to judge too. 

Putting it all together usually leads me to buying used broken-in, depreciated gear when possible and considering my purchase an extended home demo at the cost of potentially reselling effort/cost. Cheers,

Spencer

@chayro 

You just say aloud something that a lot of us have known for a long time but couldn’t articulate as well as you did. It takes me months to assess a new piece of hardware and I usually won’t know until I remove it and see if I missed it. 
There is nothing wrong with measurements. That’s where it starts but listening has to be the ultimate value judgement. We always have to make sure we are measuring the right thing and only listening can validate that. 

@spenav- thank you for the compliment, but it was Amir who actually articulated it when he spoke about the analytical brain as opposed to the music-listening brain. Like you, I have felt this for a long time, but couldn’t crystallize the idea as well as Amir did. And while we’re here, I think we should stop using the word “improvement” and substitute “the component that sounds better to me”, which is, I think, what we really mean. 

Measurements are useful to me just to understand how and why my electronics are doing what they do. But our ears are incredibly sensitive to nuances that can't be measured. I think many audiophiles have experienced the joy of adding a component that is a huge upgrade to their ears in sound quality but measurements say it shouldn't be so.

  A recent thread about Carver amps comes to mind.They don't measure as expected and many owners were unpleasantly surprised, but the sound quality to them is outstanding. Having the information is a good thing but not the most important thing.

 If I find myself stuck in analytical mode it's time for a couple of days off,then go back later and just listen and relax to a wide variety of music.

@chayro 

Great topic. You have accurately described the problems which arise when trying to assess a new component. IMO the next step is to find ways to overcome the biases and desires of our mindset. Lets face it, when we spend $1, 2, 5, 8K on an upgrade we want a return on our investment. And if one isn't there, we can easily fool ourselves into believing it IS there. I've done it. And I suspect many others have done so as well.

But you and others have given the answer to the problem. @spenav  has described my (and others) method saying,

It takes me months to assess a new piece of hardware and I usually won’t know until I remove it and see if I missed it.

This seems to be the method of you and most every poster so far. I  don't take months to assess an item as others have said, but weeks for me. And like spenav,

I usually won’t know until I remove it and see if I missed it.

That, for me, is the key. I believe it is easier to see or hear what we've missed than to hear what was gained, though that is in a mental database for comparison