Low Frequency Rumble from TT between songs

Some tube or tubes in your SE amplifier is/are microphonic. Tube damper(s) might help to reduce the magnitude of the  noise to tolerable levels.

I had the same thought about microphonic tubes but I swapped in a whole new tube set and no change. I should borrow another tube amp just to test that as well although I'm pretty excited about the First Watt SIT-3 I just bought.

Tubes of certain types tend to be microphonic.  For example, every 6SN7 is microphonic to some degree.  The trick is to find those that are less microphonic than average, or better yet to move your amplifier around the room.  In any case, it is not totally surprising that a new set of tubes does not definitively cure the problem.  Tube dampers can help too. Also, First Watt are great, be happy.

This sounds very much like a worn bearing issue.  I was pleasantly surprised when I had my VPI bearing worked on some years ago.

I'm guessing you are writing about a used Aires.  Am I correct?  Do you know how old it is?  The bearings of VPI TTs need to be worked on on a regular basis.

First is the need for lubrication.  Is your bearing inverted?  Or non-inverted?

Next is the possible need for the replacement of the ball and thrust plate.  Have you inspected these?

Just to follow up, the issue was isolation. I was able to get rid of the noise completely with a set of MNPCtech feet and some small rollerblocks under the motor pod. 

Also the First Watt is not as pleasing as a SET amp, no way. It does a lot of things really well but it left me missing the 300Bs.