Please explain how the noise of the switching of a full wave rectifier in a power supply can pass back through the primary winding of the power transformer and through the power cord and end up on the AC mains.I can answer this question - what happens is the spike is transmitted both via radiation and by conduction through the transformer. Power transformers do have a bandwidth and it is surprisingly wide. I don't believe I have ever measured a line isolation transformer with less than 100KHz of bandwidth, although they are not always flat.
So, the diode switching on and off creates a current surge when it turns on and another when it turns off. Given we have inductance in the transformer as well as the line itself, the current starting and stopping abruptly causes a spike in noise on the line. That spike is sent out through the power cord and will affect anything susceptible nearby on that line. Some amplifiers designs have components to squash that spike internally when it is created, others do not.
The voltage spike is dependent upon the change in current divided by the change in time times the inductance. That is, the bigger change in current when the diodes turn on or off creates a bigger spike. The change in time is dependent on the turn on or turn off time of the diode, determined internally by the design of the diode. The inductance is both the transformer and grid together.
erik_squires is right, one does not want that noise coupling into audio equipment. If one has a transparent power cord connected then that equipment has to deal with the noise. If one has a power cord connected that doesn't conduct the noise, then the equipment doesn't have to deal with it.