Don’t bother. As Erik alluded to, if there’s a problem you’ll hear obvious distortion occurring on high volume musical peaks.
To perform a proper measurement, you would need a measurement device that is extremely fast acting; that has bandwidth extending beyond 20 kHz; and that provides a peak/hold function so that brief dynamic peaks will be held long enough for you to see them.
Some oscilloscopes can do that, and probably also some particularly sophisticated voltmeters. But why bother, when the onset of obvious distortion figures to be the bottom line?
Regarding the comment about the maximum output voltage of the preamp, that specification should be thought of as the preamp's maximum output voltage **capability**. And it should be, and almost always will be (at least in the case of reasonably well designed audiophile-oriented equipment) much greater than the voltage that is required to drive a power amp to its maximum power capability. Otherwise a risk would be created that the preamp itself might clip on high volume peaks, in addition to other possible adverse effects such as increases in preamp-generated distortion.
Regards,
-- Al