@gryphongryph Always, I suspect. There are a few things to consider in the balanced vs. single ended (RCA) world.
You really need to know if the equipment internally balanced by design.
If so, then you’re getting reduced distortion because of the noise rejection properties of the design scheme. While this is great for getting rid of noise, it also throws away all even-order harmonics in the distortion profile, leaving only odds. If you read about 2nd harmonic distortion, you’ll see that it actually creates an impression of warmth, body, and 3-dimensionality... and is preferred by some -- but not all -- listeners. For fun, Nelson Pass has actually made a H2 generator that literally injects negative phase second harmonics into the signal path. Like I said, not all ears hear this the same way, though, so you may prefer 3rd order harmonics (or neither), in which case it makes sense to minimize distortion and go fully balanced throughout the system.
So, if the equipment is fully balanced, then you’re throwing away the benefits by using the RCA outputs. If it’s single ended by design, then using the XLR outputs means that the signal is being pushed through a transformer or op-amp circuit to convert from single-ended to "balanced". In this case, you’re not getting any of the noise rejection benefits of the balanced circuitry. Instead, you’re getting whatever the sonic signature is of the added iron or circuitry. I’d say that’s an obvious compromise, but some people do like the sound of a high quality transformer, so perhaps you’ll find it a pleasant addition.