This statement is a common myth, and being a myth, is false. A center tap degrades the Common Mode Rejection Ratio, on account of the simple fact that no center tap is really centered exactly. For this reason, center taps are never used. This is true of dynamic mics, cartridges, tape heads and any other inductive source including input and output line transformers.
I never said that center tap would be better for noise rejection (though it absolutely could be). You appear to have have misinterpreted my use of center tap to imply a single ended ground reference, which is not the same as connecting the shield to a center tap to balance the signal w.r.t. the shield. That it is not used in audio (not needed) does not mean it does not work. It is used in other industries because it does.
https://www.pulseelectronics.com/wp-content/uploads/2016/12/G019.pdf
https://patents.google.com/patent/US20170131427A1/en
Certainly noise rejection is one aspect, but so is the rejection of ground loops (that is why the balanced line system ignores ground).
Grounds loops ARE common mode noise. Hence, the accuracy of my statement, to reduce common mode noise.
The problem here is that in order to obtain the benefit of balanced operation, your equipment must support the standard, AES48. Having introduced balanced line operation to high end audio, I can tell you that hardly any high end audio gear supports the standard and so you hear differences, and often degradation.
We are talking input to phono stages, i.e. cartridges, so not sure where you are going bringing this up. It is not applicable.
At this point we need to be pedantic since terms that are not interchangeable are being used that way, "balanced" refers specifically to the interconnect method, i.e. the signals are impedance balanced to the ground/shield, hence the noise on one is balanced and opposite to the other. That could be implemented with transformer coupling and shielded cable, or differential signalling and a shielded cable.
What is called a "balanced" input is balanced in the sense that it connects to a balanced interconnect, but electrically it is differential. The AES standard is a differential signalling standard coupled with a balanced interconnect.
Since the cartridge is already a balanced source, its a matter of getting the signal to the preamp without involving that signal with the ground system (the tonearm tube and turntable).
Which makes this statement as noted before inaccurate. A cartridge is not a balanced source. It is a floating differential source. If you twist the wires and maintain a consistent (w.r.t. each wire) shield to the amplifier then the interconnect would be balanced.
Because the cartridge is floating, and hence its output differential, going into single ended input, assuming you use the practices of balanced interconnection (with shielding), you can still have a low noise floor especially at low frequencies, because the common mode noise at those frequencies will be rejected. However, from a practical standpoint, this does not work well at high frequencies, due to parasitics. This is where the differential input comes in that will reject common mode not just at low frequencies but at high frequencies.
Unfortunately, too often balanced and differential are thrown around interchangeably and they are not and mean different things. This is confusing especially if you are trying for a deeper understanding.