Best practices when conducting a DAC comparison

@david_ten 

As I mentioned in my 11-06-2017 1:17pm post, "I ensured the volume control on my amp never changed". So while I do not disagree with George, it could be a moot point and you may want to forgo the hassle of precise level matching. Why, because the difference among the DACs may (and likely should or will) be audible from a tonality, dimensionality, or other characteristic that can be readily detected irrespective of level matching. Personally I'd "worry" about precise level matching only when and if you're at the stage when all else is equal. 

"I ensured the volume control on my amp never changed"
  it could be a moot point and you may want to forgo the hassle of precise level matching.

All dacs have different analogue outputs levels even if it's by a 1/10th of a volt, how can anyone compare dacs sound, if the levels are not matched, even a minute increase in level will favour that dac in an A/B comparision.

Cheers George  

All dacs have different analogue outputs levels even if it’s by a 1/10th of a volt, how can anyone compare dacs sound, if the levels are not matched, even a minute increase in level will favour that dac in an A/B comparision.

Hi George. You are right. I have no disagreement with regard to the different output voltages. In my case, I compared the maximum output specification of the Yggdrasil (4.0V RMS (balanced), 2.0V RMS (single-ended)) and that of the Oppo UDP205 (RCA) 2.1±0.2Vrms. (XLR) 4.2±0.4Vrms) and concluded that for my purposes the output voltages were close enough to not be a concern.

I knew my comparison would be less than scientifically exactly perfect, however, I did want/desire as accurate results as possible *and practical*. Practical being the operative word, and precise level matching may not be practical or necessary.

I disagree that a minute increase in level well *necessarily* favor a DAC in a A/B comparison. It may, or it may not. What I’m merely suggesting to the OP is that there are other factors besides "voltage output levels" that matter even more when doing the A/B. For instance, the Oppo treble is more "tinny" than that of the Yggdrasil. That is true regardless of the output voltage, and one doesn’t need to jump through hoops and level match to find out.

Obviously one does not want to listen to DAC "A" at 60db and DAC "B" at 100db. Sure, if the *result* of the output levels produce egregiously different volume then something is amiss and that could/would invalidate a test. But a 10th of a volt, all else being equal, I think not.

EDIT:

Should the OP have the proper equipment to precisely level match the output voltage, certainly there is nothing wrong in that and in fact that would be preferred. So in no way am I suggesting not to level match. Instead, I'm calling attention to the merit of doing so relative to other items and the likely hassle involved in attempting to *precisely* match.

@georgehifi  and @gdhal  Thanks for your thoughts on the level matching point you are making and are mainly in agreement on. 

@stfoth  I like the 'enjoyment' part of your assessment. Your primary suggestion on how to go about the evaluation is valid and has good company in that it has also been offered up (similarly) as an approach by a couple of others.