This is a consequence of system gain structure. You have a very high sensitivity speaker, with a lot of combined gain from your preamp and power amp AFTER the preamp’s volume control. In net, you have way more gain than you need, and you're hearing the noise floor that comes along with generating all that gain. The preamp's volume control can't save you here because it's UPSTREAM of all that gain. You wouldn’t hear so much "rush" noise if any of the following were true:
- More of your system’s gain was apportioned to BEFORE the volume control (but then you might risk overload conditions from "hot" sources)
- You insert attenuators between your preamp and amp (e.g. -10dB), raising preamp volume to compensate
- You use a preamp with either lower gain OR a higher signal-to-noise ratio. Many solid state preamps have a higher signal-to-noise for the same gain as tube amps, but there are some tube pres with very high ratios - like ARC Reference. My VAC Master is also pretty good here.
- Use an amp with less gain / lower sensitivity
- Use speakers with lower sensitivity /efficiency
I have the same issue with 96dB Tannoys (at 2.83V input = 1 Watt), tube preamps with "less than amazing" signal noise ratios and > 10dB gain, and power amps with mid-upper 20dBs of gain or more. Usually of the two - tube pre versus tube power - the tube preamp is by far the bigger culprit for noise, in my experience.
A lot of tube preamps and tube power amps have a lot of gain, because there are only so many circuits, and most of them use a front-end stage with lots of gain. 12AU7 and 6SN7 provide enough gain as is, but something like 12AT7 or 12AX7 gets crazy. The highest gain tube amps I’ve had started with 12AX7. And yes, microphony can be a big problem with many tube preamps too.