Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

I have a few products that ASR have trashed and a few they most likely would and they all sound great.   

The other night I wanted to go to a local restaurant....  my lady said " I heard that place is horrible"   I said " who told you that, it's great "    She said "Chris"    I said " I thought Chris is a vegetarian?"    

This hobby has far too many variables to make choices without listening to the gear you want to buy.   

Buying based on ASR recommendations is kind of like having a vegetarian tell you that a steak house is bad.   

@jssmith , arguably it does especially since after recording the RF radiation and sound at the same time they simulated the RF radiation and no sound was heard. A few tried and no sound was heard. Correlation was assumed to be causation and it turns out that was not the case. The current theory is it is a thermal effect from visible and IR radiation. Scientists have recreated the measured levels and have been able to generate an acoustic effect similar to reported incidents.  I understand one issue with the RF theory was lack of an identified receiver that could generate an acoustic effect, especially true as the effect has been reported for centuries dating back to a time when very little was made of metal.

@bobitto

We can go opposite way: if audio waves are unmeasurable and so sophisticated for current audio measuring equipment, how companies then produce the equipment then? Do they measure the results at all? Or maybe they just use alchemy, philosophic stone and any other magic when engineering the equipment to reproduce audio for our fragile ears :). Sure no need to measure, just price it $1,000 000 and it will instantly become the best “audiophile” product or the year, lol.

That’s not how it works. Of course they measure the audio products they build - during and after the design process.The most important things are a waveform of the output stage (null test), measuring with an oscilloscope, and a select few measurements that go beyond just the standard 5 or so. SINAD is an outdated way to measure audio equipment. Yet it is used as a gold-standard on ASR.

Then you must also consider the cost of parts, paying their employees, advertising costs, office space etc. There is obviously a lot that goes in to running a business.

Common sense tells us that for a hundred bucks, we shouldn’t be able to get a DAC with superlative performance, but ASR (Audio Science Review) tells us of course we can!

The word "Science" in the website should hint at a hypthosesis for why audio gear meant for the same purpose sounds different; and should welcome 3rd party testing - like other real scientists. However, that is not allowed over there...just try to challenge the results - suggest further measurements. Open the device up. Take a picture of internals and indentify the parts used. Do a reliability test. None of those things are done....not to mention countless errors in testing.

We actually do have fragile ears - since once hearing loss is an avanced stage (bad genetics or excessive exposure to loud environments) we can’t get it back. Using a hearing aid is not fun. I have some family members who unfortunately lost their hearing in one or more ears...

Human hearing (in particular the sensitivity; to detect changes in tonality) is incredibly advanced. As such, we can hear small differences easily; providing our hearing is not compromised.

I have a few products that ASR have trashed and a few they most likely would and they all sound great.

Smart design choices. They built a machine that could provide great sound quality; and left out meaningless specs like 0.0000000000002 % THD and SINAD as a yardstick measurement...

Buying based on ASR recommendations is kind of like having a vegetarian tell you that a steak house is bad.

Golden statement...

Totally different impressions and MEASUREMENTS on Head-Fi for the same product. @amir_asr likes to suggest that his "instrumentation" is so much more accurate than what others are using. Well with that logic, upgrade every 3 months or whenever AP releases a new flagship audio analyzer. This means that every former product was substandard or less accurate in some way. Right? Check out my profile to see a photo as proof of this IEM review...

And the way he EQs headphones is painful to see. It makes me furious. He simply drags up/down a line on a log EQ so it inherently influences the frequencies around that octave as well; rather than fine-tune with proper notches in place and compensate with a preamp option in the software so the levels are not compromised. You’re welcome @amir_asr

Then he has the NERVE to message me on here and tell me that I can’t hear a difference. Nobody is golden-eared - sure, however if we have good hearing and are trained listeners with knowledge, we can certainly discern all kinds of differences in audio equipment.

he says:

"And on topic of the above research showing how poor people like you in discerning differences in speakers let alone electronics:"

Also bad grammar. I am not poor, but he is suggesting that "people like me" do poorly at discerning differences in speakers and other electronics...people like me...lol

@oddiofyl

Therefore your sensory input comes first.

So how do you sort through the thousands of audio components?

amir_asr

 

I'm all for objectifying audio if it is possible and correlates to what we hear. Speaker FR is one example of that.

May I ask what are your thoughts on listening as a way to evaluate a component? Do you ever listen to some of the components you measure and wonder why you hear a difference and why measurements don't explain it? Why something may measure good and sound bad?

This is not an attack on you. But internet may come across that way. I am  Just curious.