Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Showing 2 responses by realworldaudio

When not handled properly, obsession with A/B tests is just as any another obsession turned into compulsion: harmful and self-sustaining. It ultimately stems from 1., lack of self confidence 2., lack of experience, 3., trust issues.

With these issues solved, it becomes meaningless whether you do a change blind or seeing. (Plus, you need follow up for it to work - break in period, different genres, susceptibility to line AC changes, etc, so just swapping as a form of A/B test is shallow at best.)

When listening to music / system, I just open myself to experience without pre-judging the system. Why? Because I heard cheap solutions punching heavy weight and ultra-bucket mortars turning out as duds, and one piece that sounded like a rust bucket singing in another system. I learned that we cannot bring a single preconception to a listening experience, and I have zero trouble accepting that.

So, go in with zero conceit and preconceptions. Seeing the box that I will hear will bias me to judge it AFTER the hearing on a different scale. (Should have done more or punched high above the weight).

After all you only truly need an AB test when you are a MANUFACTURER and you want an edge to sell.

 

Aural memory is the result of a learning process, and an ability that we can develop just as any other cognitive function. Improving it is the same as learning a new skill.

When there is effort put in, it can be vastly improved upon, and conversely, lack of training will result in the decline of aural memory faculties. 

Just as we can remember words, colors, pictures, we remember sounds with analogous mechanisms. A person who can retain a 7 digit number for 10 seconds can train to withhold it in memory for an hour or longer, eventually retaining the ability to call back the number after a longer time period, and with increased digit counts.

Same with aural memory.

As every learned skill, it takes a lot of effort....some people have a head-start, but ultimately perseverance is key, and is the secret to a good aural memory.

A/B test cannot substitute an acute aural memory.... perhaps with an A/B selector switch we can cut out the time-caused retention-loss... yet, in case the aural perception and memory is so poor that it fades in seconds, then the detail observation skills are also comparatively abysmal. In consequence, the A/B tester with poor aural skills does not have the cognitive faculties to perform the test in a competent way.