Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

@thespeakerdude

That is because we do know how to translate measurements into what is heard and it is far less variable.

You can’t tell how a speaker sounds simply by how it measures. There is 0 possibility or we would all by buying speakers from mail order catalogs of graphs. More proof you don’t have a system, and are making stuff up ad lib.

 

 

@tsacremento 

  1. Does a unit’s chain-of custody or provenance impact your confidence in the results of a unit’s test results being truly representative of the model?

For the vast majority of cases, no. Most of the reasoning was explained in my last response.  Let's remember that if there is an issue with Golden samples, it would affect other reviewers far more than me because they exclusively get samples from manufacturers.  In my case, a large number of products for test come from members.  A good portion of these are purchased new and drop shipped to me.  And large percentage of used ones are current products.  I occasionally test vintage products or discontinued ones because they are available on the cheap on used market.  Manufacturers are welcome to challenge the results of any used product tested but I have yet to encounter one.

@tsacremento 

  1. How audible will the difference between an electronic component producing 0.1% THD and one producing 0.001% THD be when played through a transducer generating between 1.0% and 2.0% THD?

The nature of distortion is different in electronics vs speakers.  For example, speakers have no self-noise (passive ones anyway) whereas electronics do.  This is why you can put your ear next to a tweeter and hear hiss and buzz.

On pure distortion front, speakers produce their most distortion in bass region where we are not critical anyway.  An amplifier distorting will do so at all frequencies.  Have an amp clip and you can hear it on any speaker even though it may not reach the bass distortion of said speaker.

There are stated of the art speakers and headphones that have clocked 80+ dB SINAD which is the limit of what I can measure for them. 

That said, detecting non-linear distortion is not easy for most listeners. So you could say that maybe people can't hear even elevated distortions 1 to 2%.

Here is the thing though: the only reason a piece of electronic generates 0.1% THD is due to sloppy or bad ideas in design.  It is almost never the case that it is done to make the equipment cheaper.  Indeed, by far, the situation is the reverse: you pay far more money for a gear with more distortion and noise!  You pay more to get more noise and distortion.   Yet you can buy a device from companies that care that have provably inaudible noise and distortion for very reasonable cost.  We learn about this by measuring.  If we didn't, we would be going by marketing words of expensive gears and not objective, reliable data.

 

@alexatpos 

Now, who am I to tell them, or to anybody that they are ’wrong’ ?

Do you live in a frame of mind where there is no knowledge, everything is only ever "opinion" so we can't ever explain why someone is "wrong?"

I hope not.  But if you don't...why do you think for a moment it is in principle "rude" or nobody's place to explain why a claim is likely incorrect?  Why can't one person point out another person may be in error?

But, if you say that my (or of many others) point of view (or hearing) is the fruit of my imagination than I would reserve the right to consider you rude...with exception to of all that people who are not mad enough to even consider getting involved in hi fi in the first place.

Which is just continuing the very problem I described in my post to you.

You are taking any suggestion of your fallibility as "being rude."   Why would you do that?   Aren't you fallible like anyone else...are do you consider yourself infallible?

If you don't consider yourself, and your perception, infallible, why would you consider the suggestion you MIGHT be wrong to be "rude?"  

That would be like in my example of my son in the scientific study - the study takes for granted our fallibility and seeks to account for it - no mature adult takes 'offense' at the idea they could be wrong, which includes...yes...imagining things through various biases.

When I bring skepticism to your claim to hear differences between USB cable I am not simply declaring that you are wrong and that you imagined differences.   It's more nuanced than that.  It's that the claim is inherently technically implausible given how USB and digital signals work, so some level of skepticism is warranted (and the reply to that skepticism would be to explain how the expensive cable would plausibly alter the signal!)

I'm also keeping in mind that we are all fallible and prone to perceptual biases, where we can misinterpret things - for instance misinterpret changes in our attention to sound, which can produce a different impression, as coming from the gear rather than how our brains work.

I have experienced this a number of times myself - experienced "hearing obvious differences" when I knew which gear was playing, but when controlling for sighted bias, those sonic differences were not detectable.

So I'm not declaring that you "didn't" hear a difference.  Only that it is an area of well known controversy and that you COULD have been subject to a bias effect, like any other human being.  Which is why I personally will prefer to wait for more reliable evidence for such claims.  You don't have to.  I'm explaining why I do.

Again, if you see my bringing forth these issues of all too human bias problems as "rude" then you may as well be declaring your own perception as infallible and out of bounds for questioning.

So...why be like this?  Why take the very possibility that you might be wrong as a personal insult?  It doesn't get us anywhere and it's unnecessary and misunderstanding the nature of the conversation and intentions.

 

 

 

 

 

@prof is putting forth very valid arguments ... You may take issue with his response to other’s beliefs as is your right but can you fault his methodology and back that up?

I’ve quoted the ill logic he increasingly employs. There can be no better proof than that!

This is his latest use of ill logic in an attempt to conceal his previous use of ill logic:

You know very well the point I was making ...

Nope, it’s not my job to try and read his mind.