Caps do a lot of thing, but unfortunately in tube preamps they are a necessary evil we wish we could avoid.
The biggest issue with caps in a tube preamp are the caps used in the signal path, and are almost never neutral or transparent enough, so good ones command the big bucks, and some makers will provide "special editions" of a model which are identical except for the caps. Conrad Johnson and PrimaLuna are examples, but may add other part upgrades.
The coupling caps must is block the DC voltage that's always present at the output of a tube while letting the music through untouched. The DC is around 300V. Obviously, you don't want your amp, designed for 1-2 Volt signals getting the high voltage bias, so you have to use at least 1 coupling cap at the output. These are small farads (2-4uF) but high in voltage. They must be high enough to block DC even if the full voltage of the power supply should be applied, which is possibly around 4-500 Volts. The physical size is proportional to the farads x voltage (more or less). So, a 4uF cap at 600 V may appear enormous.
Then, for fancy caps, the higher the price, usually the larger and more complicated they are physically. Some use serial construction, a hugely wasteful (of uF) design where you put one cap after another in the same tube. Trouble is, each cap must be double the final size. That is, for a 2uF cap, you build 2 x 4uF. That is, you create a cap with 8uF of which you'll get 2 out. I have no idea if this is actually better, but it's the basis for the highest end Mundorf among others.
Best,
Erik