Does it pay to upgrade tubes in power supply of preamp??

I would expect the 6V6 and 6SL7 to have more effect.

The 6V6 is the pass regulator tube. The 6SL7 is the voltage feedback tube and controls the 6V6. So you can see that both would have a greater effect than the rectifier, which is upstream from both of these tubes. Essentially the voltage from the rectifier has to pass through the 6V6 to be regulated before getting to the preamp itself. The output of the rectifier is a sawtooth waveform that is smoothed by filter capacitors; after the 6V6 regulator there should be no sawtooth waveform present in any quantity at all.

Replacing any tubes can have an effect upon the sound if I was you I would contact Brent Jesse as he has  a lot of knowledge and can steer you in the right direction all the best

+1 for Brent Jesse. I've bought several vintage Telefunken tubes from him, the last time in 2012 and highly recommend his service.

Before buying any tubes you should give your new preamp and it’s tubes time to break in. Once this occurs and you know how your new preamp sounds then go tube rolling. There is good advice above as to what to try.

Upgrade them asap and everything will sound much better.