Could I suggest, respectfully, that you perform a couple of very important techniques.
First, the differences between servers will be material but at this level they are each very, very good. The fact that all you are changing with speaker setup is a little toe-in speaks volumes. With each material compnent change in my system comes a complete re-setup of the speakers. Once they are marked on my fllor then, and only then, can I reasonably compare the two presentations.
The Zenith is not at the same performance level as the N20, even with the Phoenix. It doesn’t mean you shouldn’t like it more but it might also mean that with the Innuos chain being completely optimized for USB and the Gryphon dac board is certainly optimized for USB and your speaker setup optimized for that chain…which brings us to the N20.
Second, you might consider pumping the brakes and excerising more patience. Wait for the proper aes cable to arrive and settle in, take the time to reposition your speakers properly, take good notes. Mark the floor (not with a sharpie but properly notated painters tape LOL). Revisit and repeat. You are about to introduce a third and potentially fourth set of server/streamer variables and you are doing it at breakneck speed. Decide between the two high quality server/streamers you have on deck. A-B til you are sure which you like better and then, sell one on and demo/purchase the next one in the same manner. Then and only then will you possibly get where you want to go. Live with something a while.
Depending on the day, a Bentley is a more satisfying drive than a Range Rover…on other days, the opposite….over a longer period of time….you will prefer one over the other when all things are considered. Listen to bad recordings of good music…which setup makes that exercise more fun.
Perhaps the most important thing to inject into the process? While the inboard Gryphon dac board is quite good, you seem to be seeking a 10 out of 10 with a steady stream of $10-30k streamer/servers….they will each sound very different, especially if your speaker setup remains reasonably static. It isn’t a race so allow your right brain to guide the experiments and your left brain to evaluate the musical enjoyment.
As always, best wishes with the journey.