Audio Science Review = "The better the measurement, the better the sound" philosophy

@westcoastaudiophile

I have couple questions to “ASR Tests Originator”:

1: Can you please describe your measurement equipment' employed at ASR LAB? I assume tester is calibrated, maintained, and upgraded periodically, - correct?

You assume wrong. We are not interested in metrology, or measuring things with repeatability to 5 decimal places. Typical SINAD (noise+distortion vs signal) varies by a 1/10th of a dB as the gear is sitting there.

That aside, call Audio Precision and ask if their analyzers require calibration. They will tell you that only if  you are doing government work and such and need such "cover your behind" certificates.  Measurements I perform are routinely replicated by manufacturers and other third-parties.  

That aside, I have a long list of equipment in the lab. Key ones are:

Audio Precision APx555. This is a $28,000 state-of-the-art analyzer. It is used for testing DACs, headphone amps, amplifiers, pre-amps, phono stages, etc. I have been a customer of Audio Precision since early 1990s when I worked at Sony.  AP is the gold standard in audio measurements.

Headphone measurements are based on APx555 above with addition of GRAS 45-CA measurement rig ($14,000).

Speaker testing is performed using Klippel Near-Field Scanner from German company. It is a robotic system for measuring anechoic response of a speaker in 3-D space. It costs roughly $100,000.

These are the core instruments. Beyond them, I have a ton of other gear from scopes to meters and everything in between for specialized testing.

I would say if you had to put together a lab like mine, you would need about $200,000 of capital investment plus tons of specialized knowledge and experience.

- are you doing your measurements in Faraday Cage?

-what cabling, loading, power, additional filters, shielding, etc are used for ASR tests? ..and why do you think your test setup is good to represent average user?

2: How do you select units for tests? Do you receive devices from manufacturers, or do you obtain those randomly?

No Faraday cage. Those are used for EMC testing.  None of you  use your stereo gear inside such a cage so I don't see why I should test them that way.

Cabling for XLR is Mogami Gold. For RCA, I use Amazon Basics. No power conditioning is used or needed. I have tested a ton of these and either do nothing, or impair performance. I do have a LAB AC generator that I use for special testing.  Ditto for DC lab generator.

Vast majority of audio gear that I test comes from members or those that I personally buy. Manufacturer gear is 5 to 10% of the total. I routinely test duplicates sent by members and have yet to see a case where the company sent unit is golden sample. A number of companies also offer that test units be purchased randomly if there is lack of trust.

My oscilloscope doesn't listen to music.

@fleschler

Amir-again dumb retort. Of course good designers use test equipment to design and test their results. However, if it ended there, they are making HUGE mistakes without knowing it.

Well, the dumbest thing is to take the word of a designer attempting gorilla marketing in forums and such. Why on earth would you trust what he has to say about his own gear? Isn’t he totally biased to get you to buy his gear? Would you trust a drug from a drug company with no government certification and controlled testing? Why would you put your guard down when it comes to audio? What happened to common sense?

And no, they would not make "HUGE" mistake if they stopped at instrumentation. As I explained, there is no reason to know they have good ears whatsoever. On what basis you put any value on what they think of their own gear vs another? He knows how to design electronics but all of a sudden, he has golden ears too?

Nothing wrong with claims about fidelity. But that better be in controlled listening tests published with full documentation. Him saying he tried many things and the one being sold is the best is hogwash.

I tested a PS audio directstream DAC. It measured poorly in low frequencies. After a bunch of back and forth with claims you believe in, it turns out that for cost reasons they used low quality transformers that produced more distortion! Multi-thousand dollar DAC used low quality parts. So you better not believe company claims. Your trust should only be on what you can verify. And I am here for verification. If you don’t like that, setup your own controlled listening tests. These are the only proof point that respected audio journals accept. Don’t make up your own methods. They only serve to confuse you into spending money after bad.

@invalid

My oscilloscope doesn't listen to music.

Then you don't know how to use it or it is broken.

@fleschler 

 Speaker designers who don't listen to adjust settings are not very good designers.

I am sure the person I was responding to was talking about electronic design. Of course speaker designers listen to their speakers. Key is whether they do that using controlled testing or not. And whether they use proper set of measurements in addition to that.  If the answer is that they don't use such measurements and only use their own ears to figure out what is or is not good sound, it would be a speaker I would stay away from.