Measurement Bias

Measurement bias is the idea that if you know the measurements of something that information will already bias your perceptions of it’s performance. For instance, knowing the g-force at which car A might slip vs. car B could affect your perception. B, having a higher g-force rating should be a "better" car but if you did not know this you might rate A as better. It may be more fun to drive.

I’ve seen this happen in a review in (I believe) TAS. The review was for a DAC I purchased. The reviewer noted it as "noisy." I read this after I had purchased and been listening to the DAC for a while. I was more than a little shocked, I could hear no noise whatsoever from the DAC. I pulled out an oscilloscope and sure enough, there was unexpected ultrasonic noise on the outputs. I eventually did sell the DAC, but not because it was at all noisy, but because new DACs handled Redbook (44.1kHz/16 bit) tracks so much better. Nowhere in the review did the reviewer mention they had measured it, and they certainly did not point out the deficiencies in Redbook playback, but the reviewer absolutely presented this DAC as noisy but otherwise OK.

So, my point is, that making assessments on the experience that comes from a measurement in audio is tricky business, and if the reviewer is aware of the measurements ahead of time it will absolutely bias them into hearing things which they’d otherwise not, and leave them blind to other real world challenges.

If you want to put together desirability with measurements you need to look at the work Floyd Toole or Bose have done and others in this area and you wont’ find it in a frequency response chart. Of course, Bose’s research is proprietary, but absolutely no one one earth has spent more money on assessing value vs. measurements and manufacturing dollar than Bose.

PS - Please don't argue the quality of Bose speakers here.  I'm not arguing for or against them.  I'm arguing that the research done in tying together desirability and engineering direction is outstanding.  That is all.


stereophile does it right.  reviewers don’t know the measurements until their review is complete.


In this particular aspect, I agree, though I have seen JA completely misread his own measurements, that's another story.

I dont understand the apparent need for validation by measurements. It should simply be enough to like the sound of a component and all this mention of bias is just masturbation. It is impossible for us to know the motivations of others in terms of the choices they make when they buy equipment just as we cant know what bias may or may not exist.

When mijostyn says that the "design speaks volumes" he is right on the money. If you let measurements lead the way you will most likely end up with S.S. devices and speakers which adhere to particular design criteria. Rather than asking if a device measures well you should be asking how did the designer achieve these measurements and what sacrifices were made to achieve their desired results. 

I have two amplifiers ( out of many ) that did not fare so well with Amir's testing, but I love listening to both of them. Enjoy ! MrD.


I dont understand the apparent need for validation by measurements. It should simply be enough to like the sound of a component...

That’s exactly where I stand. We listen with your ears and brain, so listening is an excellent way to determine what gear you like the sound of. (sort of seems like a "Duh" moment to me! 😉) Manufacturing audio gear is a different story, but choosing what you like the sound of shouldn’t require measurement validation any more than determining that a steak tastes good, or that a sports car looks nice in red.

There is a measurement bias for sure... There exist also a techno cult strong bias...The next religion will be techno-cultism... Those implanted and those Amish...

Guess which one i want to be ? 😊