Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

@nyev 

@amir_asr , thank you for sharing your perspectives. I have an education in computer engineering. But being an audiophile, I just don’t agree with the position that the science and measurements can totally explain our perceptions. I have a fundamental belief that science cannot explain all of the dimensions that impact our subjective interpretation of physical sound waves. Why? You have suggested folks get upset when ASR rejects a component that they subjectively praise. I think you are correct in many, many cases. It’s why people get so fired up about ASR. In my case I don’t care if ASR rejects a component that I subjectively enjoy - that doesn’t bother me in the slightest because if I enjoy it that’s all that matters to me. So why do I follow my subjective judgement over science? I simply don’t believe that science can FULLY explain, at our present level of understanding, how sound waves are subjectively interpreted by humans. 

First, thank you for kind attitude in asking this question.  :)  Much appreciated.

On your point, it is very true that we don't understand why we perceive what we perceive.  Advancing that knowledge though is domain of neurologists who want to diagnose disease.  The science we follow is psychoacoustics which is the "what" we near and don't hear.  If needed we do, we do draw from neuroscience but in general, we don't need to.

Example, we know most people can't hear above 20 kHz. The why has to do with the design of ear.  But we don't need to know that.  We simply conduct controlled tests and find out the highly non-linear frequency response of our hearing.  And then we use that to build things like lossy audio codecs (MP3, AAC, etc.) which work remarkably well in fooling people into thinking they are hearing high fidelity sound.  Again, we can look at features or of our hearing like IHC, filter banks, etc.  but we don't need to, to build a loss codec.

By the same token, we can measure a device's electrical characteristics and then determine if they fall below our threshold of hearing.  Once they do, the what is what matters, not the why.  We can declare the device transparent.

Now, I should note that listening tests play a huge rule in audio science. Every speaker and headphone I test relies on decades of psychoacoustics and controlled testing to develop the target responses.  Again, it doesn't why we are the way we are.  In speaker testing for example, we all seem to like a neutral response even though we have no idea how the music was mixed and mastered!  We have an interesting compass inside us that says deviations from flat on-axis response is not preferable.  

To be sure our hearing is complex.  For example there is a feedback loop from the brain to the hearing system to seek out information in a noisy environment.  This is the so called "cocktail party effect" where we can hear people talking to use even though there is so much background noise with others talking.  The brain dynamically creates filters to get rid of what you don't want to hear, and hear what you want to hear.

This causes problem in audio testing.  You listen to product A. Then you go and listen to product B, hoping to find a difference.  Your brain obeys your orders and will tune your hearing differently.  All of a sudden you hear a darker background.  Details become obvious that were not.  None of this is a function of device B however.  It is happening because you have knowledge of what you are trying to do, and use it to hear things differently in a comparison.

Due to understanding of above, we perform testing blindly.  Once you don't know which is which, your brain  can't bias the session.  Actually it tries but we run enough trials to find out if that is a random thing, or due to actual audible differences.

So as you see, we understand what we need to understand to determine fidelity of audio products.  Said products are not magical.  They have no intelligence  Measurements as such, powerfully tell us what they are doing. 

 If frequency response is flat and independent of load, has distortion and noise below threshold of hearing, then you can very confidently declare is transparent.  This analysis assumes perfect speakers.  To the extent the speaker is not, then the job gets easier and hence the reason amps with noise and distortion above threshold of hearing are also declared as transparent.

Why then SS amp always has a haze where as tube amps always has a transparent sound but we all know tube always have inferior frequency response vs. ss.

In short you can't measure it.

 

In the debate about science vs senses as a way to evaluate audio equipment…

Both work.  And neither works.

@andy2 

Why then SS amp always has a haze where as tube amps always has a transparent sound but we all know tube always have inferior frequency response vs. ss.

In short you can't measure it.

Oh we can and do.  Measurements say that they don't have haze.  And proper listening tests confirm the same.  That you think otherwise means you are in dire need of performing a controlled test where you don't know which is which.  Otherwise, your bias as stated above will always give you what you want to hear, pun intended.

In my testing of many tube amps, they either have low enough distortion to sound just like solid state devices.  Or, they have sufficient distortion to sound muddy at lower volumes and then get quite distorted.  No way are they transparent in any form or fashion.  Now, if you are not good at hearing such non-linear impairments, then and you operate under the above bias, then your conclusion will be what you stated.

See, all explainable.

@tcotruvo 

In the debate about science vs senses as a way to evaluate audio equipment…

??? Audio science heavily embodies listening tests and so includes our senses.  It however does that correctly by testing just what we hear, not what we pick up with other senses.