Strage problem - power lines amps /preamp noise

So for the longest time I thought I have had bad luck with some used amps that I have been buying here... While my current amp; SF Power 1, sounds good, it is also very quiet (no hum, no noise from the transformers, powers up clean w/o any pop etc..). Several other amps, both tube and ss, would make noises; loud pop during power up, hiss and noise thru speakers etc.. I was guessing it may be power but just over the last few days of experimentation I realized that it is a bit more complicated.
So again; my current system (SF1 to Cary SLP-03) makes no noise whatsoever. Once I have a different amp connected I hear a white noise / hiss thru speakers with left channel being more pronounced.
What I have discovered is that if I disconnect my preamp (not just power off but unplugged) - the hiss is no longer present. BUT...I can also eliminate the hiss by simply plugging the preamp to another outlet, HOWEVER, after an hour or so of playing the noise would come back and then all I have to do is move the preamp to the previous outlet and the noise would disappear again...but again only for about an hour.

All outlets I tried are on the same circuit... I also tried (just for the heck of it) computer UPS unit with the same effect.

What gives? Any idea if there is something wrong with the preamp? My power lines? Combination of both?

Thanks!