Audio Science Review = "The better the measurement, the better the sound" philosophy

@decooney

The most respected audio designers listen more with their ears than a graph.  

How on earth would you know the list of audio designers and the "respected" subset?  And how do you know that is a truth as opposed to a talking point post poor measurements of their gear?

I can also say the opposite. How would anyone know if you are right or I am?
 

I will answer: you need to understand electronic design. If you did, you would know the above claim can't possibly be true. Modern electronic design starts with simulation and full analysis of circuit performance including things like distortion, frequency response, etc. Then prototyping starts with the designer's eyes glued to instrumentation like voltmeters, oscilloscopes, and if we are lucky, audio measurement gear. If they lack the latter, heaven help us as they have no idea if they are designing something performant!

But let's say you are right. So what? I am supposed to trust the ears of a Joe designer?  They could have all the design expertise in the world. It doesn't mean they have critical listening skills and know how to compare audio gear sound without bias in controlled listening tests.

Bottom line: you are falling for marketing lines. Demand proof that their equipment is well engineered and transparent to the source. Anyone can say anything.  Ask for reliable, third-party verification. Don't worship heroes.  Insisting on validation.

Amir-again dumb retort.  Of course good designers use test equipment to design and test their results.  However, if it ended there, they are making HUGE mistakes without knowing it.  Speaker designers who don't listen to adjust settings are not very good designers.  My example of measurement uber alles for making the most expensive and as good as it should be at $850 million orchestral hall just confirms that there is more than measurements in design.  After most modern orchestral halls are built (including that one and my local Disney Hall), they nearly always undergo significant renovation to make them sound better.  Why, because we hear the results and adjust afterwards which can include new measurements and cannot rely on only scientific results. 

@westcoastaudiophile

I have couple questions to “ASR Tests Originator”:

1: Can you please describe your measurement equipment' employed at ASR LAB? I assume tester is calibrated, maintained, and upgraded periodically, - correct?

You assume wrong. We are not interested in metrology, or measuring things with repeatability to 5 decimal places. Typical SINAD (noise+distortion vs signal) varies by a 1/10th of a dB as the gear is sitting there.

That aside, call Audio Precision and ask if their analyzers require calibration. They will tell you that only if  you are doing government work and such and need such "cover your behind" certificates.  Measurements I perform are routinely replicated by manufacturers and other third-parties.  

That aside, I have a long list of equipment in the lab. Key ones are:

Audio Precision APx555. This is a $28,000 state-of-the-art analyzer. It is used for testing DACs, headphone amps, amplifiers, pre-amps, phono stages, etc. I have been a customer of Audio Precision since early 1990s when I worked at Sony.  AP is the gold standard in audio measurements.

Headphone measurements are based on APx555 above with addition of GRAS 45-CA measurement rig ($14,000).

Speaker testing is performed using Klippel Near-Field Scanner from German company. It is a robotic system for measuring anechoic response of a speaker in 3-D space. It costs roughly $100,000.

These are the core instruments. Beyond them, I have a ton of other gear from scopes to meters and everything in between for specialized testing.

I would say if you had to put together a lab like mine, you would need about $200,000 of capital investment plus tons of specialized knowledge and experience.

- are you doing your measurements in Faraday Cage?

-what cabling, loading, power, additional filters, shielding, etc are used for ASR tests? ..and why do you think your test setup is good to represent average user?

2: How do you select units for tests? Do you receive devices from manufacturers, or do you obtain those randomly?

No Faraday cage. Those are used for EMC testing.  None of you  use your stereo gear inside such a cage so I don't see why I should test them that way.

Cabling for XLR is Mogami Gold. For RCA, I use Amazon Basics. No power conditioning is used or needed. I have tested a ton of these and either do nothing, or impair performance. I do have a LAB AC generator that I use for special testing.  Ditto for DC lab generator.

Vast majority of audio gear that I test comes from members or those that I personally buy. Manufacturer gear is 5 to 10% of the total. I routinely test duplicates sent by members and have yet to see a case where the company sent unit is golden sample. A number of companies also offer that test units be purchased randomly if there is lack of trust.

My oscilloscope doesn't listen to music.

@fleschler

Amir-again dumb retort. Of course good designers use test equipment to design and test their results. However, if it ended there, they are making HUGE mistakes without knowing it.

Well, the dumbest thing is to take the word of a designer attempting gorilla marketing in forums and such. Why on earth would you trust what he has to say about his own gear? Isn’t he totally biased to get you to buy his gear? Would you trust a drug from a drug company with no government certification and controlled testing? Why would you put your guard down when it comes to audio? What happened to common sense?

And no, they would not make "HUGE" mistake if they stopped at instrumentation. As I explained, there is no reason to know they have good ears whatsoever. On what basis you put any value on what they think of their own gear vs another? He knows how to design electronics but all of a sudden, he has golden ears too?

Nothing wrong with claims about fidelity. But that better be in controlled listening tests published with full documentation. Him saying he tried many things and the one being sold is the best is hogwash.

I tested a PS audio directstream DAC. It measured poorly in low frequencies. After a bunch of back and forth with claims you believe in, it turns out that for cost reasons they used low quality transformers that produced more distortion! Multi-thousand dollar DAC used low quality parts. So you better not believe company claims. Your trust should only be on what you can verify. And I am here for verification. If you don’t like that, setup your own controlled listening tests. These are the only proof point that respected audio journals accept. Don’t make up your own methods. They only serve to confuse you into spending money after bad.