Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

@1extreme 

Right on.

1extreme

They have so little regard at ASR for actually listening to components that they don’t even bother listening to them.

Exactly. The measurements and data have little value if not correlated with what we hear.

ASR does wield a lot of power and influence and can hurt a company producing good audio products.

I really don't think ASR has much power or influence at all. It's just a noisy group with grievances, which is very common today.

Basic measurements are only a benchmark, an objective standard, but how something SOUNDS is purely subjective and has to take into account intangibles like combined elements in the system, the room acoustics, speaker placement, and the listener, right?

Speakers are my thing so let's talk speakers. A reviewer does a review of a speaker. That review will tell you every piece of audio equipment used in the speaker review, probably what they had for breakfast and if they had a bathroom visit that morning. Most of that information will be useless to determine how that speaker will work for others. Missing from the review will be room dimensions, what specific treatments are in the room, the exact speaker and toe in and listener location though the latter may have some vague description. Also missing is usually the reviewers preference around imaging and soundstage. There are trade offs.  Basically most listener reviews lack all the information you need to understand how that speaker will behave in your chosen listening space.

Give me a full test set and I can provide far more useful information about how it will behave in your room, options for toe-in, problem areas for acoustic treatments, options for boundary reinforcement and treatment in small rooms, amplifier compatibility, realistically how loud it can be played. 

@1extreme 

They have so little regard at ASR for actually listening to components that they don’t even bother listening to them. Amir performs all these tests and I don’t believe he listens to a single one or even has a reference system to drop the component in to get an impression.

You need to put aside "belief" and substitute reality.  I listen to huge amount of audio gear as part of my testing.  Every speaker, headphone, headphone amp and even some audio tweaks such as power conditioners, cables, etc. have listening tests.  This adds up to about 200 devices a year that get reviewed with listening tests.

I have a reference system as well where equipment is tested.  For example, I used RME ADI-2 Pro ($2000) driving a $4000 Dan Clark headphone to test a headphone amp.  Cables are tested the same way if they are interconnects and such.  Ditto for power conditioners, etc.  Here is an example from this week alone, a headphone amp: 

"This is a review, listening tests and detailed measurements of the Eleven XIAudio Broadway balanced battery operated desktop headphone amplifier. "

Far field devices like speakers get tested in and against my nearly $100K main audio system.  Power is provided by $20K in amplification for example.  I list these prices not that they should matter, but I suspect matter a lot to you as to what makes a "reference system."

But no, I don't listen to everything.  Measurements so powerfully describe the performance of such devices, showing impairments well below hearing for example, where it makes no sense for me to listen to them.  And proceed to make up stuff like subjective reviewers do.  Mind you, if measurements show a problem, I do listen even in that category.  Here is an example of that, the PS Audio DirectStream DAC: 

https://www.audiosciencereview.com/forum/index.php?threads/review-and-measurements-of-ps-audio-perfectwave-directstream-dac.9100/

"I started the testing with my audiophile, audio-show, test tracks. You know, the very well recorded track with lucious detail and "black backgrounds." I immediately noticed lack of detail in PerfectWave DS DAC. It was as if someone just put a barrier between you and the source. Mind you, it was subtle but it was there. I repeated this a few times and while it was not always there with all music, I could spot it on some tracks.

Next I played some of my bass heaving tracks i use for headphone testing. Here, it was easy to notice that bass impact was softented. But also, highs were exaggerated due to higher distortion. Despite loss of high frequency hearing, I found that accentuation unpleasant. WIth tracks that had lisping issues with female vocals for example, the DS DAC made that a lot worse."

You could have figured all of this out by simply going and looking.  But instead you just repeated incorrect talking points.

@cleeds 

Exactly. The measurements and data have little value if not correlated with what we hear.

They absolutely are correlated with what you hear based on science of psychoacoustics which is entirely based on listening tests.  Problem you all have is that you don't to hear, pun intended, the consequences of that.  Instead you want to live in a fantasy world where differences that don't exist or are below audible, are so obvious that your wife can hear from the kitchen.  That kind of made up effect is psychological and can never be caught or explained in a proper, science based, review of audio product.