This is where volume control quality comes into play. Cheaper potentiometers will often be poorly matched between Left / Right channels especially at lower volume levels. The ubiquitous Alps RK27 is OK but maybe not great here, at least when you get up to the level of that Cary (I don’t know what the Cary uses, but there’s a good chance it’s this). This is why the insane Alps RK50 goes for such a premium. If you open it up the Alps RK27 is easily recognizable by its sandwich of 2 or 4 navy blue plastic "gangs" (they used to call this part "Blue Velvet") and a metal can motor & controller at the end (optional for remote volume control). The RK50 is a huge beautiful cylindrical brass beast (it won’t be this) and requires an external motor rig with belt drive to be remote controlled.
On the flip side a good stepped attenuator is utterly transparent sounding and well matched at all levels, but the steps themselves are too big on the lower end of its scale so it can be impossible to find the "right" volume when you have too much gain.
Digital volume controls give good matching and good granularity, but some will claim they don’t sound as good as the other 2 types.
Besides channel matching, the other problem with high gain preamps is that your music signal will be riding closer to the noise floor anytime you’re not utilizing most of its gain. So you’re much more likely to hear a hissss noise floor. The suggested attenuators will help BOTH your channel matching and signal to noise ratio.
The problem with some tube preamps is that it’s very easy to provide "way too much" gain for use with modern digital sources, with many tube circuits. Look at some of 1970s/80s ARC/ CJ tube preamps with ludicrous 20dB+ gains and 12AX7 tubes - this was before the digital age really took hold. Most balanced DACs push out 4 Volts. That’s way more than required to slam most amps well into clipping without ANY added gain from a line stage.