Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

That’s not how it works. Of course they measure the audio products they build - during and after the design process.The most important things are a waveform of the output stage (null test), measuring with an oscilloscope, and a select few measurements that go beyond just the standard 5 or so. SINAD is an outdated way to measure audio equipment. Yet it is used as a gold-standard on ASR.

Can you tell what those 5 select other measurements? You appear to put yourself forth as an expert, so as opposed to a vague statement, should you not detail that information?  I mainly go to ASR for speaker reviews. Professional interest. SINAD is not even part of those measurements, probably because speakers don't make noise. Because they are using a Klippel system, their measurement suite is far more extensive than say Stereophile. What is published is extensive and is similar to what we would measure internally. We have proprietary weighting functions for some measurements. I would not expect a review site to have that.  I counted 11 graphs on a DAC review, in addition to SINAD. I picked a DAC since that seems to be most discussed here.

 

Common sense tells us that for a hundred bucks, we shouldn’t be able to get a DAC with superlative performance, but ASR (Audio Science Review) tells us of course we can!

Whose common sense?  Asian manufacturing costs, Asian parts costs, a DAC a model or two down from the top, high volume, low margin business model. In our speakers with digital in, the DAC section, with enough performance to have no audible impact, is not expensive. How do we know there is no audible impact? Measurements and blind listening tests.

 

The word "Science" in the website should hint at a hypthosesis for why audio gear meant for the same purpose sounds different; and should welcome 3rd party testing - like other real scientists.

Have you noticed that ASR is publishing reviews including extensive measurements from people other than ones done by Amir. The science in the website would indicate that proving something sounds different would be a necessary first step.

 

However, that is not allowed over there...just try to challenge the results - suggest further measurements.

Here I will agree. Amir can be quite defensive and arrogant. He is not as open minded as he should be.

 

Open the device up. Take a picture of internals and indentify the parts used. Do a reliability test. None of those things are done....not to mention countless errors in testing.

Do reliability testing? Can you name even one review website that does reliability testing? Do you have any idea of the cost and time required?  I saw on several reviews pictures of internals. Not all, but a lot. What percentage of audiophiles are able to accurately assess the internals of a product?

Most of the speaker tests do not have errors. Some of the more esoteric speakers I feel have errors in testing. Some of the claims errors I have seen are more sour grapes. Given the volume of testing, errors are to be expected. Do you think other test sites are perfect? Some of the explanations and tests I see done by others, especially with lesser equipment make me shake my head.

 

Totally different impressions and MEASUREMENTS on Head-Fi for the same product. @amir_asr likes to suggest that his "instrumentation" is so much more accurate than what others are using.

Have you noticed that numerous reviewers are popping up and using guess what, the exact same equipment as ASR. Imitation is the best form of flattery I guess. His equipment does appear superior to most traditional sites. Headphone testing is very hard to do repeatedly to address that specific issue.

 

Well with that logic, upgrade every 3 months or whenever AP releases a new flagship audio analyzer. This means that every former product was substandard or less accurate in some way.

This is not a logical statement. As the test gear currently is accurate enough to identify artifacts we may hear, further accuracy only plays to marketing specifications, not audible artifacts. What is required now, is better tools for interpreting measurements, not better measurements.  For speakers, there is room for improvements for measuring distortion over emission angle, but that applies to everyone.

 

And the way he EQs headphones is painful to see. It makes me furious. He simply drags up/down a line on a log EQ so it inherently influences the frequencies around that octave as well; rather than fine-tune with proper notches in place and compensate with a preamp option in the software so the levels are not compromised. You’re welcome @amir_asr

I will leave it up to others to make their determination if your statement is true of not.

Interesting take. I’ve heard meteors as well (one over Tampa, FL around 1973 that was seen and heard by tens of thousands of people that night and I was at an outdoor concert - the meteor was LOUD).

But, yeah, “SCIENCE!” said…

NOW do a little online searching for validity of eyewitness accounts of accidents.

It’s really simple, right? Four people (count two of them - the drivers - as participant/observers too) witness a collision at a 4-way intersection.

Which story is correct? Police and claims adjusters and courtrooms are often employed to adjudicate these things because people’s memories are fungible things; what I see isn’t necessarily what you see based on your perspective and focus and abilities to recall what you saw (or think you did).

It’s been 60 years since JFK was assassinated, and eyewitnesses insisted on shots from other areas of Dealy Plaza and the grassy knoll and people are still arguing about it today. 

Basic measurements are only a benchmark, an objective standard, but how something SOUNDS is purely subjective and has to take into account intangibles like combined elements in the system, the room acoustics, speaker placement, and the listener, right?

A lot of this boils down to simple opinion, and everyone has one of those.

Amir does Amir, you do you.

@1extreme 

Right on.

1extreme

They have so little regard at ASR for actually listening to components that they don’t even bother listening to them.

Exactly. The measurements and data have little value if not correlated with what we hear.

ASR does wield a lot of power and influence and can hurt a company producing good audio products.

I really don't think ASR has much power or influence at all. It's just a noisy group with grievances, which is very common today.

Basic measurements are only a benchmark, an objective standard, but how something SOUNDS is purely subjective and has to take into account intangibles like combined elements in the system, the room acoustics, speaker placement, and the listener, right?

Speakers are my thing so let's talk speakers. A reviewer does a review of a speaker. That review will tell you every piece of audio equipment used in the speaker review, probably what they had for breakfast and if they had a bathroom visit that morning. Most of that information will be useless to determine how that speaker will work for others. Missing from the review will be room dimensions, what specific treatments are in the room, the exact speaker and toe in and listener location though the latter may have some vague description. Also missing is usually the reviewers preference around imaging and soundstage. There are trade offs.  Basically most listener reviews lack all the information you need to understand how that speaker will behave in your chosen listening space.

 

Give me a full test set and I can provide far more useful information about how it will behave in your room, options for toe-in, problem areas for acoustic treatments, options for boundary reinforcement and treatment in small rooms, amplifier compatibility, realistically how loud it can be played.