For the last 4-5 years I pursued my system upgrades and find that the best sound comes from components and accessories which work well together. Having said that, I think that having better quality earlier in the process is a good thing, so spending more on a DAC might be better than spending more on the "down line" components.
I had my eyes opened to what a better DAC could do, when I dipped my toe into the "higher end", buying my first SACD player, whose internal DAC made my regular CDs sound better, capturing much more detail than I ever knew was on those CDs.
About two years ago I bought a stand alone DAC (PS Audio DirectStream DAC), and that unit still puts a smile on my face.
As for the price, to say a $32k DAC is "worth the price", only you can make that judgement.
One thing I've found, shopping on Audiogon, is that there are people out there who seem to like to try new things, do so for a few months, then offer them on the site, where you can snag some good values. I've bought a few things that way, and haven't been disappointed.