Best practices when conducting a DAC comparison


Reaching out for general advice on how Agoners have compared DACs in their own systems.....

....and how you have determined the 'better' or the 'preferred' component, based on your comparison.

This will be my first in-depth comparison.

Feel free to mention whatever you believe will help and stuff I may need to look out for / be aware of.

Thank You.
david_ten
Since the question will be asked:

Denafrips Terminator

Schiit Yggdrasil (my current primary DAC)

Exogal Comet Plus

and if I have the time and energy an Oppo 105 and a Wadia 121.
With Roon it is easy, just play the same track to two grouped zones (two DACs) and then switch inputs on a preamp to hear either one. Just make sure you level match with test tone.
It is not an easy thing to do, because the differences will be tiny at most. The methodology is simple, however. First, look at measured test data to find if there any weird anomalies. Second, if you are then still inclined to do listening tests, make sure that levels are very very precisely matched (to witin 0.2 dB). The best way to do so is by using a good volt meter. Next, switching should be near instant, given humans' short audio memory of at best a few seconds. Finally, comparison should be blind, to avoid expectation bias, and should be repeated multiple times, to exclude random error.
It is the unwilingness to submit to such a rigorous test protocol that generates the audiophile fairy tales of night and day differences between DACs and other electronics. People hear differences and believe that what they hear is real. But that is not necessarily true. Good luck - it is a hard job.
I can also predict that those not trained in scientific research methodology will immediately pounce on me.