Amir and Blind Testing


Let me start by saying I like watching Amir from ASR, so please let’s not get harsh or the thread will be deleted. Many times, Amir has noted that when we’re inserting a new component in our system, our brains go into (to paraphrase) “analytical mode” and we start hearing imaginary improvements. He has reiterated this many times, saying that when he switched to an expensive cable he heard improvements, but when he switched back to the cheap one, he also heard improvements because the brain switches from “music enjoyment mode” to “analytical mode.” Following this logic, which I agree with, wouldn’t blind testing, or any A/B testing be compromised because our brains are always in analytical mode and therefore feeding us inaccurate data? Seems to me you need to relax for a few hours at least and listen to a variety of music before your brain can accurately assess whether something is an actual improvement.  Perhaps A/B testing is a strawman argument, because the human brain is not a spectrum analyzer.  We are too affected by our biases to come up with any valid data.  Maybe. 

chayro

Well, at least I feel better knowing it isn’t just me getting that vibe from ASR.

I watched one of his videos a year ago. He tested some expensive speaker cables and dismissed them as comparing no different to lamp cord. His followers seem to eat these types of videos up. I challenged his conclusions with first, was his analyzer calibrated and traceable back to the Bureau of Standards. His answer was no, and that calibration is not necessary since it was a new piece of equipment.. Anyone who publishes data with uncalibrated equipment is immediately suspect. Secondly, anyone who publishes data without the information needed to duplicate the testing makes the data meaningless.

That was as far as I got before getting endless abuse from his followers so I moved on. He tends to test everything with his new (last year) analyzer. To someone with a hammer everything looks like a nail.

@tonywinga, there has not been a Bureau of Standards for almost 35 years. Bringing this up shows you don’t have any experience in calibration or measurements which is likely why you were "abused". Other than absolute gain, what exactly do you think a NIST traceable calibration would be on an audio tester? A large portion of the test gear for what I do is not NIST traceable for calibration. Why? NIST does not have any standard remotely applicable. It’s fine for basic things like current, voltage, weight, thickness, etc. but not for complex measurements that either don’t have a standard or are defined by other technical bodies.

This is why I don’t post often in audio forums. Conversations just end up as pages of whining because 1/2 the world does not believe what I believe and I am offended that they will not stop. What is worse, people will make things up to support their argument.

Take this piece below. This is so factually incorrect that I wonder if the author even visited the site? I don’t go to ASR often either, as most of their threads end up as technical fights about things I don't care about, but when I was planning my room, I asked a lot of acoustic questions there and I can state without reserve, that most there would never discount acoustics and would abuse you if you said acoustics did not matter.

 

Their common point is a total ignborance of acoustic condition because they need electrical tools they dont need to tune a room... All rooms are equal for their activities....
 
I said it pages ago, this will be just whining until one side proves to the other they are right, but I am beginning to think people would prefer to whine instead of trying to reach a conclusion. Even if either side conclusively proved the other wrong, I am not sure they would accept the outcome.

Wow, another self proclaimed expert.  Spectrum and frequency analyzers do need to be calibrated.  Since I was an Engineering Manager for several years with one of my responsibilities being test equipment design, calibration and repair in a major Aerospace Company I know that first hand.  All test equipment should be traceable back to the Bureau of Standards or NIST for serious test work or analysis.  Nitpick over the name if you like.  And guess what- equipment loses its calibration.  Protocols were required to deal with product previously tested on equipment found to be out of calibration by more than a certain amount.

I use my iPhone or iPad apps to measure things but I treat is as comparative data, not absolute and I always qualify that as reference only.