Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

@nyev 

I know the argument remains unresolved - are these things in my mind, or is it that science has not yet figured out how to measure certain things that we perceive in audio?

We can't measure what you perceive.  You listen to a track that brings you joy.  We can't measure joy.  What we can measure is the sound coming out of your audio gear.  That is the sound that is "heard."  What is perceived includes many other variables that go beyond sound. I enjoy watching my Reel to Reel play music.  It brings me joy.  But that has nothing to do with the sound.  The sound is excellent but I hear background noise which is not so nice -- something we absolutely measure.

Put more strongly, you have to identify what is sound and only sound in your perception.  You can't look at something, and give an opinion because your knowledge of what you are seeing pollutes your perception.  A person not likely horns will dislike any horn speaker.  Put them in a blind test though, and they won't bring such preconception to the party.  To wit, I was shocked how good these JBL horn speakers sounded in double blind tests at Harman that I took:

So bottom line, bring us an experiment where only the sound is evaluated and shown to not be random outcome and I will show you a measurement for it.  The moment you include other things, we can't measure it due to no fault of science, measurements or engineering.

And oh, there is no science being developed to determine what you state.  The science is completely settled that only results of controlled audio tests matter. where only your ear is involved.  All else is not worth even looking at.

@amir_asr , I thought you had given up on the discussion and I would not at all have blamed you if you had! Thank you for responding. Below are three examples where I experienced subjective differences in sound and I genuinely wonder if you would be able to measure the differences I was perceiving. If your conclusion is that no, these perceptions are due to mental bias, that will not offend me one bit. I will disagree with you if that is the case, but I won’t be offended or defensive! I would also ask, how can you be sure you are measuring the right things, the right way, to explain the differences I am perceiving? That question IS rhetorical, as I would argue that no one can answer that question conclusively.  I don’t expect to convince you and I don’t expect to be convinced, but I’m interested in a constructive dialogue simply to explore perspectives differing from my own.

Just to give one recent example to relate to my prior post: Last year I bought a second USB cable (Audioquest Diamond) while I had the original for a few years. The new one sounded inferior. To the degree that I wondered if there was a design change or changes to production. But after a few hundred hours, I could no longer tell them apart. Yet existence of burn-in is endlessly debated.

Another recent example is what I found with length of USB cables, where a Nordost Valhalla 2 2m cable sounded superior to the equivalent 1m cable. Intuitively I would have expected the 1m cable to sound better, as I had not at that time read the theories, to my knowledge unproven, that USB cables should be longer than 1.5m to accommodate “reflections”.

And I don’t know if this one is proven or if people just have theories grounded in science, but the whole anti-vibration/isolation tweaks that really do make a surprising difference. I introduced a friend who is newly into HiFi to Herbie’s Tenderfeet, and he promptly cut up some yoga mats to replicate the benefits himself. He tells me it worked! It’s easy to identify whether the feet were added or removed, at the transition points.

I wonder, would @amir_asr be able to measure the differences I’m hearing, in each of these example cases? Not a rhetorical question; I’m genuinely curious. If the answer is no however, I think the assumption might be that it’s all in my mind. Just like what the researchers said about those hearing meteors, 25 years ago, because they were not measuring the right things or applying the right science to explain our perceptions.

I should also mention I had the exact same experience as with thenNordost USB example, testing a .7m Audioquest Diamond against a 1.5m version of the same cable.

@andy2 

That is true in theory but I don’t think Amir even doing that. But jitter is difficult to test in frequency domain, just to name a few.  There are more.

We don't "measure" jitter in time or frequency domain.  Jitter modulates the primary tone that we can play.  We perform a spectrum analysis of that tone and jitter sidebands jump right out:

See those sidebands at whopping -130 dB.  We can easily measure them by simply sampling the analog output of the DAC.  No need to probe inside which is problematic anyway as the DAC chip likely has jitter reduction.

As a major bonus, we can apply psychoacoustics analysis to the jitter spectrum and determine audibility as you see noted on the graph.

The other thing is that jitter is not one number as is often talked about.  Above you see multiple jitter sources at different frequencies and levels.  This makes FFT analysis far superior to any time domain jitter measurement that spites out either a number or even a distribution.

FYI, my audio analyzer has a time domain jitter meter but it is nowhere near as good as the above spectrum and at any rate, only works on digital sources, not analog.

@nyev 

Below are three examples where I experienced subjective differences in sound and I genuinely wonder if you would be able to measure the differences I was perceiving.

That would be jumping a step.  As I explained, first it has to be established that what you are perceiving is sound and only sound.  To do that, you need to run those tests blind and repeat 10 times and see if at least 8 out of 10 times you can tell which cable is which.  Bring that to me and I will guarantee you that I can measure that effect.  

I just reviewed three JPS Labs cables (XLR, USB and Power).  I performed listening tests on all three.  All sounded different than my generic cables.  Measurements did not show any evidence of those differences.  Why?  Because my testing was unreliable, ad-hoc subjective tests.  These tests produce all kinds of outcome for me as they do for you.  The problem as I keep saying is that our perception is so variable that it doesn't lend to reliable conclusions of fidelity differences until we put in some controls.

@amir_asr , I get your point.  I’ve done many blind tests in the past but unfortunately I didn’t feel the need to conduct them in these instances when I had the relevant gear.  I now wish I had!

I know this is not a “control”, but as I mentioned in the USB cable example, I would have expected, intuitively, for the shorter cable to sound better than the longer cable.  That was my bias going in.  I was surprised to find the opposite.  
 

I know, not a controlled test!  But I think it’s still relevant.