For me it depends on the RCA or XLR electronics and the quality of the RCA or XLR ICs not whether it is RCA or XLR... that's how I hear it :)
Unfortunately, XLR electronics remove 2nd order harmonics as part of the XLR noise filtering/cancelling process, which our ears like, but leaves the 3rd order harmonics in place including the noise, which our ears do not like... :)
If the Hot and Cold XLR electronics is not well matched then the noise filtering/cancelling process can be even stronger where even more 2nd, and even 3rd, order harmonics are removed. I guess that will provide a smoother, maybe warmer, sound :)