Never heard an amp do this before, ideas?

Take the amps to your local stereo shop to check out some speakers with it.  Gives you a reason to go listen to some speakers.  Maybe even to work with a pair of headphones.  Eliminate the environment.

You would not be using expensive speaker cables, would you?   I'm recalling back in the day, there were speaker cables that certain "tube" amplifiers did not like. Would set the amp into oscillation.  My idea was that the circuit were seeing a resonant tank circuits of their tubes, transformer and speaker cable. Try going back to zip wire or other cable. 

I second @jasonbourne52 suggestion. Does it make the noise when the inputs are connected to something and amp is powered on?

@yage unfortunately yes, inputs have no effect. 

@hobbes101 What do you have in close proximity to the amp? Satelite receiver? Cable box?

@oldaudiophile led lights on dimmers, a WiFi router, hvac, and refrigerator. A few switching power adapters plugged in throughout the apartment. Florescent lights in the bathroom.

All of these things are suspect to me, most are pretty far from the unit. I've shut down breakers and made sure lights/hvac/wifi/refrigerator are off without effect, though.

​​​​​​I'm in an apartment on the 4th floor in NYC. About 8 floors up is some Verizon equipment. Across the street there is a big service elevator.

part of me is thinking I should find a way to locate the RFI noise somehow, but I'm not sure how.