Speaking of a phono cartridge, it's just a floating coil with 2 wires. As the coil is floating, you have the choice to operate it as either single-ended or as a true balanced signal source. If you connect the negative terminal to ground, and use the positive terminal as signal, then you have normal single-ended mode.
To better understand the advantage of a cartridge working in balanced mode, I will quote a paragraph from my book "Noise Reduction Techniques in Electronic Systems" by Henry W. Ott: "A balanced circuit is a two-conductor circuit in which both conductors and all circuits connected to them have the same impedance with respect to ground and to all other conductors. The purpose of balancing is to make the noise pickup equal in both conductors, in which case it will be a common-mode signal which can be made to cancel out in the load".
According to this definition, we can confidently say that a pickup cartridge truly acts as a balanced source, because when the cartridge is connected to a normal phono cable (with 2 wires and 1 grounded shield), the impedance of both wires with respect to ground is the same. Further, the noise pickup is equal in both wires, forming a common-mode signal that a differential input stage (or a center-tapped transformer) will easily cancel out.
Please notice that differential mode is not the same as balanced mode. In fact, an input circuit working in differential mode is a pre-requisite for balanced mode. As said, this circuit could be: 1) A center-tapped transformer, or 2) a differential input stage (tubes or solid-state). As you can see, balanced mode has obvious advantages over single-ended mode in phono cartridges.