Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

The interesting experiments are really hard to do. I can imagine a test involving giving some audiophiles long term daily access to a very highly regarded system in an excellent listening room. The trick would be to change things out and see if they hear it. Change out a real copy of an amp or pre-amp or other component with a shell version that looks identical but has different innards. Have available a form to report if they hear something sounding off or degraded, or improved. They would never be told about the fake components being sneaked in on them. The point isn’t to put them on the spot, but to see if they notice changes when they aren’t expecting them at all. I would distract them with some kind of claim that the test was about medical and mental benefits of high end audio use, or something like that. If it is found that they do notice changes on gear that supposedly measures well enough to be beyond the hearing limits of humans, then we have something interesting to start testing - just what is it that’s different about these components that allows people to hear the difference? If something is found, then that can be tested in future components and our understanding of how to make good sounding equipment will be advanced. Without some testing like this, all the reporting of improved sound without a meaningfully correlated measurement is just anecdotal accounts of subjective experience that give us little to build on. People perceive they are hearing differences I have no doubt. But they don’t know why although they offer conjecture. Perhaps the manufacturers do know why but if so they aren’t sharing information that would be very valuable to science and would dramatically enhance their reputations far beyond the audiophile world. As it stands I don’t see research in high end audio reproduction trickling advancements into other fields. That kind of substance doesn’t seem to be present. It’s always the other way around. Or am I wrong?

I saw the post referenced in this thread and think, as many here, it's a valid point. After all the ears have predictable behavior but not the brain. It has all sorts of biases. I recall listening to a system in a store when the rep changed out the DAC to a different model and yes, the sound did change. In the presentation they used 'night and day' or some other statement. All I could think is that they both sounded pretty bad. Now, that's a harsh opinion, but doesn't negate his interpretation that one sounded better, and good for him, that improvement is meaningful and worth the cost. 

Read about acoustic and psycho acoustic and forget " gear listening fetichist" and their rejection of blind test and forget " measuring tool fetichist" and their embrace of blind test instead of their own ears... These oppposing camps are united by the SAME ignorance...

I am not against the use of blind test by the way...

But using his own ears like acoustian learn to use them is the key to audio...

And the materials you put on the wall and the cheap homemade device you will use dont need to have a price tag but need to have the right physical amd mathematical acoustical properties...All my acoustical devices cost me nothing and we dont need blind test to tune our speaker/room no more then we need blind test to tune a piano or to verify the tuner hearing, playing the piano will do the test...

Is it not simple to understand?

 

And dont try to test other people claims, it is a waste of time, learn how to listen in your own room...

 

😁😊

The interesting experiments are really hard to do. I can imagine a test involving giving some audiophiles long term daily access to a very highly regarded system in an excellent listening room. The trick would be to change things out and see if they hear it. Change out a real copy of an amp or pre-amp or other component with a shell version that looks identical but has different innards. Have available a form to report if they hear something sounding off or degraded, or improved. They would never be told about the fake components being sneaked in on them. The point isn’t to put them on the spot, but to see if they notice changes when they aren’t expecting them at all. If it is found that they do notice changes on gear that supposedly measures well enough to be beyond the hearing limits of humans, then we have something interesting to start testing - just what is it that’s different about these components that allows people to hear the difference? If something is found, then that can be tested in future components and our understanding of how to make good sounding equipment will be advanced. Without some testing like this, all the reporting of improved sound without a meaningfully correlated measurement is just anecdotal accounts of subjective experience that give us little to build on. People perceive they are hearing differences I have no doubt. But they don’t know why although they offer conjecture. Perhaps the manufacturers do know why but if so they aren’t sharing information that would be very valuable to science and would dramatically enhance their reputations far beyond the audiophile world. As it stands I don’t see research in high end audio reproduction trickling advancements into other fields. That kind of substance doesn’t seem to be present. It’s always the other way around. Or am I wrong?

 

Amir and co were also proclaiming Bybee devices would have no effect whatsoever on audio. Then one of them actually measured them, and the Bybee devices were reducing distortion. So did they admit they were wrong? Of course not, Amir just immediately moved the goalposts to saying the measured reduction could not possibly be audible, and then demanded double blind tests as proof the Bybee did anything. Then they got embarrassed and quickly locked the thread on their forums so it would move off the 1st page.

They probably failed to remember that Jack’s devices/similar works (works from the same mind) are being used in detection systems in nuclear submarines, for lowering the noise and distortion floor for said extremely sensitive equipment that is being used to detect underwater threats. There being likely even more, that Jack was not allowed to talk about.

I was just thinking of putting together a wide ranging (As it would have to be) article for the audio world, called:

"Synthetic psychology and the midwit problem -- in High End Audio."

Interesting ! thanks...

"Gear listening fetichist" or "tool measuring obsessive fetichist " are marketing conditioned creatures ignoring acoustic and psycho-acoustic power anyway...

In acoustic the division betweem subjectivity ande measures make no sense, because their correlation id the center of psycho-acoustic science... Acoustician are superioir audiophile because they listen sound not the gear like RCA conditioned dog listening not a room but a gramophone pavilion ... 😁😊

it is Bybee that give me the impetus to try my journey in my embeddings controls adventure search...

I bought one of his device at low cost 7 years ago... I decided to create my own heaven WITHOUT upgrading the gear but instead putting the gear in a better mechanical, electrical and acoustical environment...

I replicated the first Bybee device with great success...Using minerals +crystals experiments ...

I succeeded...It is not the same as Bybee invention but it work enough to convince me...And it work not less... But in other way than the orginal Bybee plates...

I even replicate some Bybee results about room acoustic upgrade in my own way...WHY? Because when you know that something is possible then you gain the desire, the energy, the trust you need to do it in your own way...Simple...

When you dont believe in something and you dont believe in your own brain/ears learning potential then you dont do it... Simple...

Bybee results give me confidence...

Thanks to him for his story and inspiration...

 

Amir and co were also proclaiming Bybee devices would have no effect whatsoever on audio. Then one of them actually measured them, and the Bybee devices were reducing distortion. So did they admit they were wrong? Of course not, Amir just immediately moved the goalposts to saying the measured reduction could not possibly be audible, and then demanded double blind tests as proof the Bybee did anything. Then they got embarrassed and quickly locked the thread on their forums so it would move off the 1st page.

They probably failed to remember that Jack’s devices/similar works (works from the same mind) are being used in detection systems in nuclear submarines, for lowering the noise and distortion floor for said extremely sensitive equipment that is being used to detect underwater threats. There being likely even more, that Jack was not allowed to talk about.

I was just thinking of putting together a wide ranging (As it would have to be) article for the audio world, called:

"Synthetic psychology and the midwit problem -- in High End Audio."