Digital Coax, optical, or RCA for use with my Bluesound Node player?

If you're streaming over 96kHz then I'd stop using the optical and move to digital connection.  There can be a bandwidth limitation with optical cables.  

You'll do a lot better with a 75ohm digital coax vs. RCA interconnect. What is your budget?

I've auditioned quite a few. Acoustic Zen was very good.
Cerious Technologies makes a good S/PDIF cable with a return policy.
BTW, the recommended length for digital coax is 1.5 meters. 

I picked up a Audioquest Carbon digital coax

Analysis Plus "Digital Crystal" COAX 1.5M

Continuous Cast Copper (solo crystal copper) braids, hollow oval design. Demo’d and owned various AQ, AP, Cardas in 1m and 1.5m and preferred AP Digital Crystal. Pricey, but this to a good DAC and wallah!  https://analysis.plus/product/digital-crystal/

@todd1010 So how does your new cable sound? Want to step it up again on your Node? VH Audio sells a shielded two conductor power cable in bulk that is very good. For the ends from VH Audio as well, buy Furutech plugs. I don’t have photos of my system but I did post a pic of the cable I’m speaking about. I think you’ll be surprised of the improvement a better PC will make on your Node. Well worth the upgrade.