Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

@nyev 

Below are three examples where I experienced subjective differences in sound and I genuinely wonder if you would be able to measure the differences I was perceiving.

That would be jumping a step.  As I explained, first it has to be established that what you are perceiving is sound and only sound.  To do that, you need to run those tests blind and repeat 10 times and see if at least 8 out of 10 times you can tell which cable is which.  Bring that to me and I will guarantee you that I can measure that effect.  

I just reviewed three JPS Labs cables (XLR, USB and Power).  I performed listening tests on all three.  All sounded different than my generic cables.  Measurements did not show any evidence of those differences.  Why?  Because my testing was unreliable, ad-hoc subjective tests.  These tests produce all kinds of outcome for me as they do for you.  The problem as I keep saying is that our perception is so variable that it doesn't lend to reliable conclusions of fidelity differences until we put in some controls.

@amir_asr , I get your point.  I’ve done many blind tests in the past but unfortunately I didn’t feel the need to conduct them in these instances when I had the relevant gear.  I now wish I had!

I know this is not a “control”, but as I mentioned in the USB cable example, I would have expected, intuitively, for the shorter cable to sound better than the longer cable.  That was my bias going in.  I was surprised to find the opposite.  
 

I know, not a controlled test!  But I think it’s still relevant.

@amir_asr, as an aside I will be buying a Tambaqui DAC and I have to admit it’s a comfort to see that you found it to measure well.

But if I end up not liking it, I will totally blame you for a misfire purchase.  Just kidding :)

For the record, I don’t equate the positive measurements you found to an expectation that I will enjoy its sound, in my system, in my room, with my ears, and with my brain….  How great it would be if it were that easy - I truly wish it was!

For the record, I don’t equate the positive measurements you found to an expectation that I will enjoy its sound, in my system, in my room, with my ears, and with my brain….  How great it would be if it were that easy - I truly wish it was!

Why bother measuring things if it doesnt equate with sound quality?

“Why bother measuring things if it doesnt equate with sound quality?”

@kenjit , out of interest I guess? It’s interesting to consider how a component performs, physically, in the real world, independently from human perception which is more important, IMO.

Not sure if I may have overstated my position earlier but I may have. I can admit I believe measured performance has an impact on how I perceive sound quantity. But my main point is I don’t believe it’s a guarantee of how I will perceive sound quality, because I don’t believe we have measurements that comprehensively predict this.  Amir takes the position that yes, we DO have all of those measurements.  And my question is, how can we know that when we don’t know what we don’t know?  My question is ridiculous on its own, as some have pointed out.  I wouldn’t go around questioning everything we know on that basis.  But my own subjective experiences in HiFi have given me enough of a glimpse to firmly believe that there is more that we don’t know.  That we don’t know how to accurately predict how we will each, individually respond to a component with a particular set of measured performance metrics.

To use the car analogy from above, horsepower and torque are valuable measurements but don’t guarantee one will enjoy driving the car.

I’m in favour of JA’s approach to measurements in Stereophile, where subjective listening is the focus. But it’s fantastic that the measurements are there just to see if SOME correlation with the subjective experience can be gleaned. Why? I find it interesting.

People are perfectly free to disagree with me!  It does not bother me one bit that ASR exists and I’m happy if they keep banging the objectivity drum.  Some say it unfairly harms certain brands but the subjective review sites can balance this.  Makes our hobby all the more vibrant, provided we can keep the personal attacks out of the equation.