Which is better - S/PDIF Coaxial or AES/EBU

In your situation, cobbling together BNC with RCA sounds nastier than taking advantage of the shared AES/EBU connections. Some say 75 Ohm coax is more dynamic, 110 Ohm AES/EBU more airy. Research here on Audiogon and you'll find the ideal length for coax is between one and two meters. AES/EBU offers a higher voltage connection. What really floats my boat with AES/EBU is locating the transport with my CDs and running the cable across the room to the DAC. Mogami 3173 AES/EBU cable, specifically designed for long runs, has a suggested maximum length of one thousand feet! My 25 feet terminated with Neutrik XLRs gives good results, indistinguishable from the one meter Theta AES/EBU it replaced.

Depends on the sound of the DAC. I use an Audio Logic DAC, the XLR is more detailed but the RCA is more emotional. So depending on what you prefer is the way to go with this particular DAC.

Happy Listening.

Depends on the transport, dac, and cable. The transmission and receiver chips are different for AES/EBU and S/PDIF; that's why the same transport/DCA combo usually sounds a bit different with AES/EBU and S/PDIF.