Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)


About 25 years ago I was inside a large room with an A-frame ceiling and large skylights, during the Perseid Meteor Shower that happens every August. This one time was like no other, for two reasons: 1) There were large, red, fragmenting streaks multiple times a minute with illuminated smoke trails, and 2) I could hear them.

Yes, each meteor produced a sizzling sound, like the sound of a frying pan.

Amazed, I Googled this phenomena and found that many people reported hearing this same sizzling sound associated with meteors streaking across the sky. In response, scientists and astrophysicists said it was all in our heads. That, it was totally impossible. Why? Because of the distance between the meteor and the observer. Physics does not allow sound to travel fast enough to hear the sound at the same time that the meteor streaks across the sky. Case closed.

ASR would have agreed with this sound reasoning based in elementary science.

Fast forward a few decades. The scientists were wrong. Turns out, the sound was caused by radiation emitted by the meteors, traveling at the speed of light, and interacting with metallic objects near the observer, even if the observer is indoors. Producing a sizzling sound. This was actually recorded audibly by researchers along with the recording of the radiation. You can look this up easily and listen to the recordings.

Takeaway - trust your senses! Science doesn’t always measure the right things, in the right ways, to fully explain what we are sensing. Therefore your sensory input comes first. You can try to figure out the science later.

I’m not trying to start an argument or make people upset. Just sharing an experience that reinforces my personal way of thinking. Others of course are free to trust the science over their senses. I know this bothers some but I really couldn’t be bothered by that. The folks at ASR are smart people too.

nyev

mastering92

Scientific papers are subject to peer review. Publishing the results instantly is not a great idea.

Quite so. Typically, a paper would undergo at least some informal peer review before it is even submitted for publication. When it is submitted, it’s then subject to a formal review before even being accepted for publication. When that part of the process is poorly done if often reflects poorly on the publication itself, so although mistakes happen, it’s exceptional.

There are those who wrap themselves in science, then use the pick-and-choose smorgasbord approach in practice. You see some do this with religion, too. Measurementalism is rather like religion in that it is taken on faith (there’s no need to listen!) rather than facts.

Of course other factors influence these YouTubers, too, as has been noted.

A lot of scientific work revealed to the public is not peer reviewed and all that is reviewed is method. The peer review does not consist of other people replicating the work as you have suggested. You are moving the goal posts. Even with peer review, many poor papers are published and if you are presenting new concepts there may not be any one "better than you".

On correlation between ad hoc consumer reports with little in the way of comparative analysis and measurements, there are many flaws in that statement. The most obvious being what is liked and what is accurate are often different if not almost always different. The other obvious reason is simply that people often assign better to new purchases, whether actually better, or in audio even noticeably different. Many people played the exact same thing twice will be absolutely convinced one is different. Either way, both of these are psychological effects of which measurements are immune.  I would also say using the word "mass" is exaggerated, but if you want to go down that path, there seems to be mass like of some of the inexpensive gear they have measured. 

Audibly transparent is not a catch phrase and needs no further words or explanation. It is a simple concept. When certain products achieve a certain level of technical performance they are audibly transparent.  You could make a product that measures better, but it will sound exactly the same as the one that does not. That is confirmed by the inability of people to hear a difference.  You made a comment about "quality" of parts, but this would be a prime example. An expensive part is often that, an expensive part but offers no benefit in improved sound. The change would be transparent. No audible difference.

@thespeakerdude  audibly transparent according to measurements maybe, but not in listening tests. 

A lot of scientific work revealed to the public is not peer reviewed and all that is reviewed is method.

There’s a little more to it than that. For example, the logic and premise are examined, and that step alone would disqualify a lot of the "science" as seemingly practiced at ASR.

The peer review does not consist of other people replicating the work as you have suggested.

That is absolutely true.

Even with peer review, many poor papers are published and if you are presenting new concepts there may not be any one "better than you".

Yes, both extremes exist: papers that have little or no value, and papers that have enormous value.

What really happened to Pons & Fleischmann in the early ’80s (hint: the University of Utah really effed it up) explains what can happen when science goes wrong, even when respected researchers are involved.