Audio Science Review = "The better the measurement, the better the sound" philosophy

I have couple questions to “ASR Tests Originator”:

1: Can you please describe your measurement equipment' employed at ASR LAB? I assume tester is calibrated, maintained, and upgraded periodically, - correct?

- are you doing your measurements in Faraday Cage?

-what cabling, loading, power, additional filters, shielding, etc are used for ASR tests? ..and why do you think your test setup is good to represent average user?

2: How do you select units for tests? Do you receive devices from manufacturers, or do you obtain those randomly?

- sometimes manufacturers prefer to send “good units” to third party test LAB to produce “excellent reviews”, while most of manufactured units of the same model are "so-so". I haven’t seen two analog sound units, including speakers, amplifiers, pre/eq/etc of the same model/brand with the same test results in my life!

regards

OK,how about...

I Am Science...

Where have I heard that before...?

@decooney

The most respected audio designers listen more with their ears than a graph.  

How on earth would you know the list of audio designers and the "respected" subset?  And how do you know that is a truth as opposed to a talking point post poor measurements of their gear?

I can also say the opposite. How would anyone know if you are right or I am?
 

I will answer: you need to understand electronic design. If you did, you would know the above claim can't possibly be true. Modern electronic design starts with simulation and full analysis of circuit performance including things like distortion, frequency response, etc. Then prototyping starts with the designer's eyes glued to instrumentation like voltmeters, oscilloscopes, and if we are lucky, audio measurement gear. If they lack the latter, heaven help us as they have no idea if they are designing something performant!

But let's say you are right. So what? I am supposed to trust the ears of a Joe designer?  They could have all the design expertise in the world. It doesn't mean they have critical listening skills and know how to compare audio gear sound without bias in controlled listening tests.

Bottom line: you are falling for marketing lines. Demand proof that their equipment is well engineered and transparent to the source. Anyone can say anything.  Ask for reliable, third-party verification. Don't worship heroes.  Insisting on validation.

Amir-again dumb retort.  Of course good designers use test equipment to design and test their results.  However, if it ended there, they are making HUGE mistakes without knowing it.  Speaker designers who don't listen to adjust settings are not very good designers.  My example of measurement uber alles for making the most expensive and as good as it should be at $850 million orchestral hall just confirms that there is more than measurements in design.  After most modern orchestral halls are built (including that one and my local Disney Hall), they nearly always undergo significant renovation to make them sound better.  Why, because we hear the results and adjust afterwards which can include new measurements and cannot rely on only scientific results. 

@westcoastaudiophile

I have couple questions to “ASR Tests Originator”:

1: Can you please describe your measurement equipment' employed at ASR LAB? I assume tester is calibrated, maintained, and upgraded periodically, - correct?

You assume wrong. We are not interested in metrology, or measuring things with repeatability to 5 decimal places. Typical SINAD (noise+distortion vs signal) varies by a 1/10th of a dB as the gear is sitting there.

That aside, call Audio Precision and ask if their analyzers require calibration. They will tell you that only if  you are doing government work and such and need such "cover your behind" certificates.  Measurements I perform are routinely replicated by manufacturers and other third-parties.  

That aside, I have a long list of equipment in the lab. Key ones are:

Audio Precision APx555. This is a $28,000 state-of-the-art analyzer. It is used for testing DACs, headphone amps, amplifiers, pre-amps, phono stages, etc. I have been a customer of Audio Precision since early 1990s when I worked at Sony.  AP is the gold standard in audio measurements.

Headphone measurements are based on APx555 above with addition of GRAS 45-CA measurement rig ($14,000).

Speaker testing is performed using Klippel Near-Field Scanner from German company. It is a robotic system for measuring anechoic response of a speaker in 3-D space. It costs roughly $100,000.

These are the core instruments. Beyond them, I have a ton of other gear from scopes to meters and everything in between for specialized testing.

I would say if you had to put together a lab like mine, you would need about $200,000 of capital investment plus tons of specialized knowledge and experience.

- are you doing your measurements in Faraday Cage?

-what cabling, loading, power, additional filters, shielding, etc are used for ASR tests? ..and why do you think your test setup is good to represent average user?

2: How do you select units for tests? Do you receive devices from manufacturers, or do you obtain those randomly?

No Faraday cage. Those are used for EMC testing.  None of you  use your stereo gear inside such a cage so I don't see why I should test them that way.

Cabling for XLR is Mogami Gold. For RCA, I use Amazon Basics. No power conditioning is used or needed. I have tested a ton of these and either do nothing, or impair performance. I do have a LAB AC generator that I use for special testing.  Ditto for DC lab generator.

Vast majority of audio gear that I test comes from members or those that I personally buy. Manufacturer gear is 5 to 10% of the total. I routinely test duplicates sent by members and have yet to see a case where the company sent unit is golden sample. A number of companies also offer that test units be purchased randomly if there is lack of trust.