Some thoughts on ASR and the reviews

I guess my confusion about the ASR site is that there seems to be a certain measured criteria that noise when below the audible range that in a dark room I (or any real/ or perceived audiophile) wouldn't be able to differentiate this product from a more expensive product even if the more expensive measured better. 

Noise is just one parameter of performance, albeit, an important one that telegraphs other performance specifications.  If the frequency response of one system is rolled off then you are going to hear that whether noise is a factor or not.  Indeed, passive speakers are noise-free (when you are not feeding them a signal) yet there are vast audible differences between them.

If you mean all measured impairments are below threshold of hearing in both devices, then yes, in a controlled, level matched, repeated test, listeners would not be able to reliably tell them apart.

First, wouldn't that mean the more expensive product should get dinged in a review because even though it measured better the differences weren't audible and the relentless pursuit of better measuring products is of no value?

Not by me in the review.  Here is again the Mola Mola Tambaqui review:

It got the soccer panther (equiv. of golfing panther for European products).  Here are my conclusions after it topped all the charts:

"Conclusions
The Mola Mola Tambaqui DAC shows again that just because a DAC is designed from ground up, it need not perform poorly. It is actually the opposite with it performing at the top of the class with respect to distortion and noise.

Since I am not the one paying for it for you to purchase it, it is not my issue to worry about the cost. As such, I am happy to recommend the Mola Mola Tambaqui DAC based on its measured performance and functionality."

There are non-fidelity aspects of products that matter to some people, myself included.  There is pride of ownership, looks, support, warranty, resale, etc. that are valuable purchase criteria.  The key is to not confuse those things with sound fidelity.

To be sure, there is pushback from membership on how I can recommend such expensive devices since you can get similar performance for much lower cost.  My answer to them is that they can judge the product however they want (they get to vote in the poll that way).  My job is not to assess how affordable something is.  That, the readers can and do take into account.  I provide the missing information, i.e. performance data and they get to combine it with above factors to make a purchase decision.

I should say that I do "curve the grades" a bit when testing super budget products.  I will recommend a $90 amplifier with certain performance that I would not if it cost $9,000.

Finally, there are exceptions.  But above is how it works almost all the time.

Yet ASR continues to bring in new products after the peak has already been reached based on perceptive hearing,  To me then it appears that the pursuit is not about music but bragging rights regarding the  numbers or  else there is an audible difference beyond the perceived hearing threshold and ASR members can tell the difference.

People and companies send me products and I test them.  I don't go and seek them out myself.  People want to know if the performance is still great or taken a step back. 

Almost every review of state of the art DAC is followed by a number of people complaining what the point of testing them is as they all sound the same.  So your comment about "ASR people" is wrong.  Members know that a few manufacturers are in a race to product the least amount of noise and distortion they can.

Note that as of late, more features are being added such as parametric EQ in this Topping D50 III balanced DAC:

While maintaining start of the art performance for $229.

Finally, please note that the asian consumer electronics market demands fresh products.  Anything more than a few months old is "not good anymore."  If I were running these companies I would produce 10% of the products they do but their local market demands much more.  So we have embarrassment of riches as they say.

The other inference is that ASR can build a minimal system based on perception measurements that when blind tested in Mike Lavigne’s listening room that the ASR system would sound indistinguishable or better than his current system. I find that hard to believe but then again I’ve never heard Mike’s system or Amir’s for that matter.

That should be hard to believe because it is not true. State of the art audio system will be expensive. My system is not cheap and neither is Mike’s:

The reason is to the left and right of Mike. To get incredible dynamics and bass down to subsonic will be super expensive. I am guessing his speakers weigh close to 1000 pounds! Your $500 bookshelf speaker is not going to do what his does. Physics and economic business models won’t allow it.

Mind you, we have made incredible progress toward bringing cost of such systems down. In the "old" analog days, every component of the system from source to speakers was expensive. Today with digital, playback hardware can be free in the form of a music player on a computer you already own. Some of us splurge a bit more for likes of Roon but that is not related to fidelity. This is a lot cheaper than a turntable.

DACs have massively come down in price while sharply increasing performance. If you don’t need balanced out, an $80 SMSL DAC will clean the clocks of many high-end DACs and provide full transparency to source content you are playing.

Amplification used to be another expensive factor. More power meant more expensive output transistors and inefficiency meant large and heavy heatsinks and cases. My amplifiers cost $25,000 each and weigh more than 110 pounds each. Fortunately, the era of high performance class D is upon us. For around $1000 you can get tons of power and superb, absolutely superb performance.

Speakers remain problematic. There is progress to bring the cost way down for equiv. performance. A Genelec or Neumann monitor bring accuracy and fidelity to die for. But except for the top of their range, they lack the amplification power and dynamics to do what Mike’s or my speakers do. Although the Genelec 8361A comes awfully close to my speakers for less than half the price.

There is a work around here with subwoofers. Those can bring the low-end but they also demand lots of work and knowledge on behalf of the owner to get them to work well with their room and main speakers.

Final note: there are a lot of expensive speakers that have poor design. So above is not to say that you pay more and get more. It is just that it is impossible to get the impressive sound that you can out of large/massive speaker systems at budget prices. I have listened to hundreds of audio systems at shows. The ones with statement speakers always do things that amaze me. No way do I go around and say you can spend $5K and get the same thing. Just not going to happen.

@amir_asr

Wow thank you for the detailed and constructive response to my post. It would be worth the $80 to test your assumption regarding the SMSL against my current horribly measuring PSaudio DirectStrem DAC (Which I love by the way).  It will be  here tomorrow.

I hope at least you were listening to enjoyable music while you were responding.

Cheers