Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

@invalid 

This is a subjective hobby after all, isn’t it, or do you guys just sit around and look at charts, graphs and oscilloscopes. I enjoy the music more because I don’t worry about how my equipment measures.

No, you worry about a ton of things in your system that don't matter while we enjoy music.  You think your wires may have sound.  You think your amp has sound.  You think the table you put the system on has sound.  You think your AC has sound.  You think digital sources have sound.  You think, well, you get the point.

We on the other hand, buy performant systems with confidence and sit back and enjoy it.  We know why it sounds right.  You don't.  You are forever chasing ghosts in audio.  The anxiety that comes with that must be immense.  

Ask anyone who has converted from your camp and above is the answer they give you.  While you keep upgrading, tweaking, replacing stuff to remove that other "veil" and get blacker backgrounds, we queue up another track to enjoy.

So I suggest getting off that talking point.  That dog don't hunt....

@alexatpos 

Well, I am very curious to hear any system that was assembled through blind testing of its individual components....

You could do that.  Or, if you are in our camp, use measurements to rule out audibility in many components (i.e. they are transparent).  For others such as speakers, you can rely on companies that perform double blind tests, or use research that correlates what sounds good to us with respect to measurements. I have done this across some 200+ speakers now.  The research works wonderfully.  Same mostly works for headphones as well although measurements there are subject to more variations than speakers.

Remember, the job here is not to give you 100% answer.  It is to get rid of 90% of the variability by weeding out clearly broken and non-performant gear.  The rest you can choose from and take in factors beyond performance.

Compare that to the alternative the few of you follow.  Completely unreliable listening tests.  1000 and one opinion about every gear, every cable, everything you can name.  True wild west with zero regard for decades of research into what makes sense.  

@mastering92 

I’m sure that most of these people doing reviews have a standard set of reference tracks; or at least a background/strong interest in audio; enough so to make their impressions reliable.

You are a fountain of untrue assumptions.  Do you even bother to fact check anything before just throwing it out at us?  Didn't I already show you how proper, peer reviewed research shows audio reviewers to not be remotely capable of producing reliable assessment of speakers?  Here is Dr. Olive's research again:

Do you see how Audio Reviewers are even worse than audio sales people?  If they are reliable as you say, how come they failed so catastrophically here?

They failed because they don't have trained ears despite all the gear they have listened to. 

That you declare them to be reliable means that you have not spent any time reading and understanding this topic.  You continue to shoot from the hip, throwing claim after claim without an ounce of proof.  All lay intuition meant to defend your position in audio.  Spend less time here and more time reading and learning about the topic.  Here is Dr. Olive's blog if you don't know where to get the research papers: 

 

Or watch my video where I go through this and explain it in understandable manner.

 

 

Whether or not we agree entirely with @amir_asr ’s perspectives (I do not, although I understand the rationale and logic), he is taking the time to present those perspectives and back them up, on a forum that is not very friendly to his perspective.  I didn’t expect Amir to engage in this discussion, in a community that rejects his perspectives with a degree of hostility, so I commend him for presenting his views in this environment!

I for one am reading his links, even if I still don’t agree fully.  But becoming educated on other’s perspectives never hurts.  In some cases I can even be convinced to change my perspective.  But I don’t think that will happen in this case - after all, remember, for 25 odd years scientific researchers assumed that reports of audible meteors were figments of the observer’s imagination!

after all, remember, for 25 odd years scientific researchers assumed that reports of audible meteors were figments of the observer’s imagination!

 

So...because you can point to something science might have gotten wrong, the lesson is...anything goes? If someone makes a claim that is, on the basis of the current science, outrageous, does the fact science has been wrong at points mean there’s no basis to doubt anti-scientific claims?

See...this is the mushiness of such a position. Perpetual Motion Machines are impossible based on current understanding of physics. Yet someone claiming they know a guy building Perpetual Motion Machines in his basement can also say "Don't be so skeptical!  Remember...science has been wrong before! Remember the meteors!"

Does that for even a moment give more credence to the claim someone is really making Perpetual Motion machines? Of course not. You can’t use anomalies where science got something wrong...which by the way is always corrected by science!...as if science doesn’t really have an excellent grasp on many things.

People are ripped off every day of the year by people selling things with the tag line "It Works! And Maybe Some Day Science Will Catch Up To Our Discovery. They Called Galileo Crazy, Remember!"

The reasonable approach isn’t to believe something dubious "because maybe science got this wrong." It’s rather to wait for solid evidence, if necessary scientific evidence, showing THAT phenomenon is real vs the countless propositions that are false.