Does it pay to upgrade tubes in power supply of preamp??

Get some vintage RCA or Philips JAN (military spec) tubes. They will be far superior to whatever Chinese or Russian tubes may be in it now. These are not really that expensive either. But don't diddle; these won't be made again.

I would expect the 6V6 and 6SL7 to have more effect.

The 6V6 is the pass regulator tube. The 6SL7 is the voltage feedback tube and controls the 6V6. So you can see that both would have a greater effect than the rectifier, which is upstream from both of these tubes. Essentially the voltage from the rectifier has to pass through the 6V6 to be regulated before getting to the preamp itself. The output of the rectifier is a sawtooth waveform that is smoothed by filter capacitors; after the 6V6 regulator there should be no sawtooth waveform present in any quantity at all.

Replacing any tubes can have an effect upon the sound if I was you I would contact Brent Jesse as he has  a lot of knowledge and can steer you in the right direction all the best

+1 for Brent Jesse. I've bought several vintage Telefunken tubes from him, the last time in 2012 and highly recommend his service.

Before buying any tubes you should give your new preamp and it’s tubes time to break in. Once this occurs and you know how your new preamp sounds then go tube rolling. There is good advice above as to what to try.