In an audio system the electrons are not flowing from the source to the destination component. The signals are AC - though a bit asymmetric -so about half the time they are flowing one way and about half the time the other way.
Although, I guess there could be some DC coupled SS pre or DAC (no capacitor on the output) where the signal is a modulated DC voltage. Has anyone ever measured a DC offset on a source component's output?
For shielded cables where, except for balanced connections, the shield is connected to SIGNAL ground (the outside of an RCA plug) so it is probably true that the source end should be grounded.