Ive found its very often advantageous to float the ground at one end of the signal path.
Agreed that is often the first thing to try out....it stops the microamp ground loop currents flowing between the various chassis as well as still grounding the shield to give you better RF protection.
Balanced circuits are better at rejecting any noise from the power supply but your description of it doesn't make any sense. It isn't a ground loop problem.
You need to read up on what Whitlock says about the reasons why XLR balanced is so important for ground loops. Think about how a power supply fluctuates under heavy demand and how this may affect microampere ground loops between gear (the BIGGEST problem in audio as RF noise pick up on analog audio is much less common). Then realize that ONLY ONE wire in in an RCA circuit will carry this current flow on your signal wires - IT CANNOT CANCEL (as it does in well built XLR gear and cabling).
=> this is why a BIG BEEFY power supply is NOT simply about more power...often it can be more important for how much cleaner the backgroud or canvas upon which the music plays can become...