Is your system a high end audio system or a good sounding audio system. If the later... using a switched plug is probably OK. But I feel very comfortable in going out on a limb and saying it is definitely going to compromise the sound quality of a high end audio system.
Provided it is not a tube amp, there probably isn't any downside to leaving it on unless it is a monster Class A amp.