Try the K50 next. Even though I’m a Grimm MU1 owner, a very happy one I might add, I’m running the Grimm into an external dac. As long as the Gryphon inboard dac is your dac of choice, you should consider server/streamers which allow for aes/ebu or USB outputs. That way you can do as you’ve done with the N20 and compare multiple inputs into your dac to determine the optimal input match.
You are already down the rabbit hole and you appear to be reading alot of commentary, which can be fuel to the fire of over-analyzing without knowing if what you are reading is truly applicable to your circumstances. Additionally, you seem concerned about duties etc which, while I understand the thought, I dont understand the logic….duties are peanuts when evaluating whether or not you can arrive at a solution with which you can be happy for the long run. Another option for you could also be to keep your Zenith, sell the Phoenix and get your hands on the Pulsar. For very little money (net) you gain the multiple outputs(aes/spdif/usb). Ive been told by a reviewer friend that it is an amazing performer for the $$$ with a warm, enveloping presentation. How would that perform with your Gryphon dac? One way to find out is to listen and you are already familiar with the form factor and the ease of being able to try it. The Pulsar is likely what you are after but its usb only.
If you are indeed committed to your inboard Gryphon dac and the two box footprint(streamer/server and Diablo/dac), then your limiting factor will be the Gryphon dac. Embrace the reality rather than spend $20k US on a server/streamer only to achieve something “a little different”.