Thing with sources is that you need as much voltage is needed for your amp feeding your speakers in your room listening with your ears.
Regarding volume pot setting - you got it all wrong
I'm happiest when pot is near max.
Don't get fooled by the high voltage of the balanced outputs of the PS Audio DAC, more voltage leads to more Gain. That means that not only the music signal but the noise will be amplified as well. That's one of the reasons why accuphase choose to have lower voltage on their DAC and that's the reason why the DAC 60 sounds better your ears once you equalize the volume level. Don't worry if you turn the volume knob pass 12 oclock. The preamp of the accuphase is designed as signal attenuator which means you get more of the original signal as you go higher with the volume.
Lastly, this article will resolve many of your concerns
https://www.diyaudio.com/archive/articles/160464-de-lite-amplifier-next-thread.html