Everyone is giving you their personal experience here, but you have to wonder how many ran controlled experiments, changing just one variable at a time, listening to the same set of test tracks that cover most/all important aspects of sound quality.
I also assumed a better DAC would make the most impact, but after running about 5 DACs and 5 server/streamers through my system, I found the server/streamer made the most difference. I ended up with a Grimm Audio MU1. The Grimm performs a lot of functions to clean up the analog wave that carries the digital signal, so the router and switch quality became much less important.
For the DAC, I finally picked a Chord Dave, which I was able to pick up used. The Grimm includes the equivalent of an M-Scaler, so you don’t need that.
Think of the server/streamer as the turntable and cartridge and that helps the process.