Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

To the ASR crowd:

Are you using a 1KHz tone?

Do you do some basic IMD3 test?

I mean music is infinitely more complicated than that.

@amir_asr , thank you for sharing your perspectives. I have an education in computer engineering. But being an audiophile, I just don’t agree with the position that the science and measurements can totally explain our perceptions. I have a fundamental belief that science cannot explain all of the dimensions that impact our subjective interpretation of physical sound waves. Why? You have suggested folks get upset when ASR rejects a component that they subjectively praise. I think you are correct in many, many cases. It’s why people get so fired up about ASR. In my case I don’t care if ASR rejects a component that I subjectively enjoy - that doesn’t bother me in the slightest because if I enjoy it that’s all that matters to me. So why do I follow my subjective judgement over science? I simply don’t believe that science can FULLY explain, at our present level of understanding, how sound waves are subjectively interpreted by humans. Why do I believe this? My own blind testing experiments, over decades, that offer results that are not explainable. Differences in power cords, the effect of burn-in, vibration control products etc. Sure there are theories why these things make a difference even when blind testing, even with generalized theories that are proven, but to my knowledge none are proven in the specific application of HiFi audio.

My engineering buddies think I’m absolutely nuts when I say music servers and USB cables make a difference in audio, but, with their knowledge, I can’t blame them at all for coming to those conclusions, as I have the same knowledge. But the difference is, they’ve not had the blind subjective test experiences that I’ve had, which leads me to question whether science really knows everything about the manner in which humans subjectively perceive sound waves.

Without going into details that may be taboo on this board, I don’t consider myself to be a particularly spiritual person, despite the objective evidence that our universe could be part of some grand design. But if I had some sort of personal subjective experience that clarified a particular spiritual path for me, I could see myself heading down that path. But nothing remotely close to this has happened. But in the world of audio, it has! That’s why I feel we need to go beyond the science!

What we perceive through our senses is not at all a “fantasy world” as you have suggested. That “fantasy world” is actually all that we have.  Sure, measurements can tell us what is happening in the world.  But no measurement, as of yet, can explain how we perceive it.

 

 

ASR does use a 32 tone test - or something like that...which is better than just 1 KHz.

@amir_asr

My post about your graphs (the bar graph) with different types of listners in a selective order was removed by a moderator! I’ve reached out to her a few times regarding site issues that have since been fixed! 

Was this done in error? Please consider that the content of my post tried to get the crux of an issue regarding ASR measurements. Anyway, I can post some of it again. Wish I would have saved it...wasted my time writing all that out.

@amir_asr

I’m not here to argue with you, but since you are here and alive/well, I figured that sharing some of what I believe is right is worth a shot.

Audio _ Science _ Review

Like I said before,

Would you consider having your test results/measurements verified by other industry experts? I don’t see a problem with that. If I were you, I would welcome 3rd party validation of my work. Why? because that is a scientific process. I have a few friends who are actually scientists, work in labs etc. And we’ve spoken about this process many times before. If I’m actively posting reviews (like you) I would certainly want others to credit my work as correct, with repeatble results.

Perhaps an online petition could be made public; then have audiophiles from this forum, your forum, and others shoot their shot. Either you get audited, or you don’t. It’s that simple. Would you be open to such a challenge?

If no one it auditing your work (much like at a workplace within a finance department); we can’t just assume that junior accountant is doing flawless work. A CPA would be a watchful eye, so to speak. I think ASR needs one as well. Nothing against you or the folks on your site.or even @kenjit

Let’s have a balanced back and forth about this. + 6 dB or whatever. Cheers.

+1 @nyev such an excellent post.

@mastering92 the ASR community repeatedly states that they are "right" because of science. You’re simply encouraging replication by other parties to verify Amir’s measurements and results. I’m sure logistics and extra work between Amir and other parties will be the excuse, but then honestly, unsure if the "S" belongs between the "A" and the "R."

@mastering92 can you name any other review site that provides measurements that does this "auditing"? 

Manufacturers always have the ability to respond and provide high quality measurements to refute the results including testing methodology. 

In the scientific world, publishing is normally the audit process. You publish and encourage others to replicate your results. Contrary to your assumptions I don't participate on ASR though I do read, mainly speaker reviews. As I wrote above ASR publishes reviews from other people. I also see others publishing their results measuring the same products ASR does though obviously a limited subset. ASR is obviously not perfect, but all the arguments I am seeing are sour grapes.