I have an Esoteric UX-3 player and experimented with using my Wyred dac 1 as an external dac. I found that the dac in my Esoteric player gave me a slightly laid back and much more relaxed sound. Since I play mostly rock on this player, I find it to be the best of both worlds. I use my laptop as a music server through the Wyred dac and have found it to bring out the best sound of my mostly classical and jazz music.
Personally, I would keep what you have unless you have a lot of money to blow through. You already have a better player than most people have and you may make a very small improvement in sound, but will probably have to spend northward of 5K to achieve it. Just think of how many cd and sacd's that money could buy.
Bob