Amir and Blind Testing

Sometimes you just need to take it with a grain of salt when reading Amir's measurements and his assessment. It is the same for other reviewers' comments.  I found, though, good measurement is not a sufficient condition of a well-received devices, but often a necessary condition.  For example, Schiit Bifrost 2 receives many good to excellent reviews from reputable reviewers but it does not measure well in ASR.  On the other hand, a majority of well-received devices have good measurements from ASR even the entry level ones (like DACs).  

I really like what Amir is doing on ASR. Measurements are super helpful in gaining a better understanding of what is happening when I like or don’t like reproduced sound. They help me get to where I want to be without stumbling in the dark so much, or overdoing something while another issue is the real problem. I generally find the science he goes by matches what I hear. I’m open though to the possibility that something that escapes the standard set of measurements could make an audibly important difference to me. Recently I’ve started to think I’m hearing a meaningful difference if I digitally reduce the input to my dacs, and then turn up the pre-amp at the analog stage to get the volume back up. It seems more dynamic. If that’s the truth I should be able to measure it somehow. As for why that might be, my dacs are modified in a way that is generally considered not good practice - they don’t have op-amps on the outputs and just feed straight into the pre-amp stage. I’m thinking they might be pooping out when asked to output too much voltage due to insufficient impedance on the pre-amp inputs. In any case, I think I’m hearing something as an improvement that I wouldn’t have thought would be an improvement because of loss of bit depth, but perhaps it is because bit depth isn’t the limiting problem in my case. For now I’m just enjoying the perception, but I’d like to figure out how to test this and see if I can show some measurements to back up this idea. If I can, I might want to invest in some new dacs, or a new crossover unit, or both! You know, I guess I could send my crossover unit to Amir and have him measure it! But then my stereo would be out of service.

@daveinpa "...But I couldn’t tell the difference between my 200 dollar topping DAC and simply plugging my laptop audio out to preamp rca inputs..."

I do not think Topping $200 ish or even sub $200 Dac (assuming your Topping Dac is one of the good ones in ASR) is that bad (unless the internal Dac inside your laptop is equally good). First off, the average output voltage from a laptop jack output is around 1.5 volts, slightly lower than the 2 volts of Dac output and the P.A input. So I believe, with everything else being equal, the volume output level will have to be lower from laptop->PA setting. Secondly, even you match the volume level with ext. Dac->PA setting, the internal Dac of laptop is in general less quality (Mac uses Cirrus Logic and is generally better) that results in less dynamic, less bass, less full midrange, and less almost everything as compared to entry level Topping or other well-received brand such as Smsl. When I upgrade from the entry level Dac to mid-tier (around $500), the improvement is also pronounced just like from laptop internal Dac to entry level external Dac. I think this "accerlerated return" improvement is consistent with what is shown in the ASR’s measurements.

I first heard of this web site not that long ago. It was from someone who used a quote from the site… and in my  experience (50 years) this generalization was simply  not true. So I went to the site and read a dozen or more reviews. Sorry, I found the comments and listening observations or recommendations that of no value and generally contradictory to my knowledge. 
 

I have been a scientist for over a decade and worked with hundreds of electrical engineers for decades in high technology… people that make the stuff inside virtually every electronic device you own. I love data: collecting, interpreting and presenting what the data is telling you. I have spent well over a decade doing that, I love the tools, the process and the outcome. But I quickly learned with high end audio this is a great way to lead you away from great sound.  Listening is the only really effective way of evaluating equipment. … although in all honesty he typically measures budget gear. 
 

I hold The Absolute Sound, Stereophile, and HiFi+ in high regards… particularly the first two. I have heard hundreds of components which they have reviewed over the last forty years and the accuracy of there evaluations is really astonishingly good. Not perfect, but very very close. Very trustworthy sources. 

@noske FWIW, I have a great deal of respect for scientists - I double majored in physics and mathematics.  I have no problem with the tests, but I'm glad you like the pictures!

Rigorous testing is always important in technology, where the goal is the production of identical, interchangeable units.  However, we're so far from "perfect sound" that we're still arguing over which imperfections matter and how much.  The first measurement we targeted was flat frequency response.  Nope, we're not there yet.  So what's worse - a broad suck-out of the midrange, or an extremely high-Q resonance above the range of hearing?  What if the HF causes your amp to oscillate?  What if it's at middle C instead?  Is THD better than IM?  What about TIM?

The answers to these kinds of questions depend on the observer and the other components in the system.  Just curious (I've only looked at a couple ASR reviews), but does he have different acceptability criteria for tube vs solid state?  If he measures speakers, what about planars vs. box speakers?  It's pretty easy to set up your testing standard to fail either group pretty much entirely.

As I said, ASR serves a purpose.  If it works for you, that's great!  Clearly, it's not everyone's cup of tea, though.

I prefer reading Stereophile (I'm guessing that Absolute Sound made this transition also when they hired Robert Harley).  They generally do a subjective test and comparison to reference & similar-priced gear, followed by bench testing that includes listening in an attempt to corroborate both the subjective & objective results.  They always let you know when there are differing opinions on a component.  The bad news is that the industry has gotten too big to cover in that depth, which has resulted in perhaps an undue focus on the most expensive gear.