XLR to RCA Adapter Dilemma

You are simply not gaining the real benefits from an XLR balanced connection. It is certainly possible that you still prefer the sound using the XLR output. If it is due to the gain, I think what you are hearing has more to do with a relative volume shift than an actual quality difference.

Are you able to use the connections in parallel and then switch back and forth and then use a dB meter to check the relative volumes. People generally prefer the louder source.

You might be seeing a impedance issue. It would be difficult to say which configuration is seeing frequency loss or gain.

Run a test sweep cd and use sound meter to see if you have any frequency dips or rises between the two to decide which is correct. After all that, use the one you like better.

... the sound is much fuller than when going from RCA to RCA. Could this be because the XLR output has a higher gain than the RCA?

Yes, that is normal. The sound quality has nothing to do with RCA or XLR Design, when you have both it depends on matching components and what technical quality the Designer has implemented.
RCA/XLR adaptors degrade the sound normally because they are internally super lousy wired. Open them and you will find even in very expensive adaptors super thin, ultra cheap wire. When you want a good one, contact SignalCable, they do custom order with Silver wire, not expensive and very, very good. You can also order better connectors (for signal transfer).

Unfortunately, my amp is not at all balanced, and I'm not looking to replace it since I like it just fine, and would rather save the dough. So, for now, it's adapter city for me. I may just contact Signal Cable and inquire about custom adapters. Thanks.

A lot of amps have an opamp on the input to handle the XLR input signals, then come back to the single end after this opamp. Where the rca input bypasses this opamp.

I have found this on some big hiend amps. You can imagine that the rca on these amps sound better than the xlr inputs when the levels are the same, but being an opamp for the xlr input some have a small amount of extra gain especially if the opamp they used is not unity gain stable.
These kind pseudo xlr inputs are more common than you think.

Cheers George