Faint buzz in speakers

I had a similar problem a few years ago.  In my case I had a subwoofer and I had to ground the sub plate amp to the preamp.  It cleaned it up

Some amplifiers are noisier than others, efficient speakers will bring this out. However, if you can't hear anything from the listening position I would leave it alone and enjoy your music. 

Since "Can only be heard with ear right up to drivers" it is not a "problem."

The solution to the "problem" is simple, don't put your ear up to the driver.

Is it a tube amp? if it is check the bias of

your tubes I just had that problem with

the hissing and cracking out of one speaker

checked everything but the bias and that was it. Good luck

Level of regulation and filtering in power supplies matters, this in regard to resistance to possible external sources of noise as mentioned above. And the more efficient one's speakers are, the more they may expose noise. For instance, I have 104db efficient speakers, 845 amps noisier than 300B, 845 amps run near 1000V, inherently noisier, require more extensive filtering, regulation than 300B. Quality of power supplies in equipment within any particular tube family can also differ. OP could try another amp to try to determine relative quietness of ps in present amp.