What would be an upgrade over the NAD M51 Direct Digital D/A Converter?

You will want to audition the Schiit Yggy for Redbook.

Benchmark Media Systems DAC3
used Bryston
Cambridge Audio Azur 851D
Mytek Brooklyn
Auralic Altair

If you have - say 2,000 Redbook CDs - and a couple dozen high bit rate downloads, SACDs and whatnot, then you will perhaps have the fun task of thinking about whether it is better to get a better Redbook DAC or a better DAC for 1 or 2% of your collection...

I'll be curious to hear your thoughts on the comparisons.  Be sure to address any ground loops, etc. ...

I would chat with Alan at Hi Fi Buts in Atlanta.
he carries NAD, Aesthetix, ARC, Brinkmann. primal Luna, Ayre... lots of DACs that people are mentioning...
the Yagi is direct from manufacturer - and I recommend reading the book also !!!

I hate spell check
make that...

Hi-Fi Buys 

These are great suggestions!  Thank you. I'm off to start research based on your help.

Of course every component may not mesh with you gear, but that is a tough dac to beat.....prepare to spend $$$

I use one for and number of sources and it sounds pretty good given its price....