When the tool is really build ballanced inside there is a easily difference to be heard.
For me it depends on the RCA or XLR electronics and the quality of the RCA or XLR ICs not whether it is RCA or XLR... that's how I hear it :)
Unfortunately, XLR electronics remove 2nd order harmonics as part of the XLR noise filtering/cancelling process, which our ears like, but leaves the 3rd order harmonics in place including the noise, which our ears do not like... :)