Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

@amir_asr

They can spend months and it wouldn’t make their reviews reliable. If you know what you are doing, including science and engineering of the gear and what the measurements show, you can zoom in and find issues. You don’t sit there listen to random track after random music for weeks. That tells you nothing.

More time spent on a task can net better results. This is not always true. However, when it comes to listening (headphones, speakers, DACs, amps etc.) spending more time evaluating a product before releasing a review can help in a few key ways:

1) Testing for Reliability

2) Features & Functionality

3) Overall sound quality analysis

4) Small important details

I’m sure that most of these people doing reviews have a standard set of reference tracks; or at least a background/strong interest in audio; enough so to make their impressions reliable. They were hired to do a task and might be very good at it. We have no way of knowing how much audio knowledge they have...

Tyll Hertsens - of innerfidelity (now defunct) was probably the best reviewer of headphones on the net. Like I said in one of my discussions, he took time to describe what each headphone sounded like with a particular track. He also did measurements and highlighted key areas in the frequency response or octaves where performance could have been better...while still quoting the measurements he took. And on top of all this, comparing it headphones in the same price-range with the same factor - open/closed back!

Since he went above and beyond, this gives a potential customer huge insights! You can go ahead and test with the same track. You can wait a week, be really busy, visit an audio shop, listen to that headphone (or headphones) and jot down your impressions on the notepad app your phone has installed.

Then you can compare and contrast your review with his - figure out if this headphone is worth the money for you. I’m sure that anyone who has bought audio products based on solid reviews like his will agree...

Now back to reviews...I bought a SABAJ A10h based on your review. I also bought a DROP THX789 based on your review. In both cases, not only did I find that output power was severely lacking; each of them also had their own sound signature. Based on your measurements and overall write-up of both units, a potential buyer would actually believe that each of them were a wire with gain!

Please see my profile for a photo that illustrates this. Looking inside one of these devices tells you it is cheap to build. Uses an OP amp and tons of subtractive distortion limiting - like negative feedback in a circuit. Tons of this, much like dynamic range compression in mastering will limit perceived dynamic range in a track. You’ve got to wonder how they put something together at that price-point and sold it. You can easily look up parts by just looking inside a unit and doing a parts inventory check...they are both not state-of-the-art ! lol

Alright..you can have your cake and eat it too! All I’m saying is....live and let live. Your tone and how you almost bully people into listening to you is rather rude. Hence why virtually every audio forum on the web has labelled you all kinds of silly names.

You need to stop listening to your lay intuition and embrace science of how to do such evaluations correctly. Formal testing shows long term listening to be much less revealing than instantaneous ones. See this published research on that:

I implore to you start paying attention to decades of research on what it takes to properly evaluate audio gear. The lay understanding and intuition stuff needs to go out the window.

Well, I am very curious to hear any system that was assembled through blind testing of its individual components....

Uses an OP amp and tons of subtractive distortion limiting - like negative feedback in a circuit. Tons of this, much like dynamic range compression in mastering will limit perceived dynamic range in a track.

 

This is bias likely derived from reading, not knowledge or experience. Your purported expertise is recording? Mixing?  Audiophiles often write about how superior audio recording and mixing used to be, even in the 70's into the 80's. Do you think most mixing consoles used op-amps, or discrete transistors?  

Can you clearly communicate what you think op-amps and negative feedback is doing to the signal, and why this does not show up even in complex distortion measurements?

 

Alright..you can have your cake and eat it too! All I’m saying is....live and let live. Your tone and how you almost bully people into listening to you is rather rude. Hence why virtually every audio forum on the web has labelled you all kinds of silly names.