A common problem with passive controls (IOW variable output impedance) is a coloration at settings that are less than full output.
This can be reduced in some situations by using a control that is itself a lower value, like 10K.
You can't count on all sources having an output impedance of less than 100 ohms. Many phono preamps and vacuum tube CD players have output impedances that are higher than that. This can often limit the utility of lower value passive controls. The trick is really in the setup, if you get it right you can get excellent results.
IOW its not a guarantee that the passive will work in all situations- you have to be careful. In the case of controls with higher resistance values, the source impedance that results when the control is set to lower values is really a function of the interconnect cable preceding the control, the source preceding the control and of course the control itself.
This is of course true of a volume control installed inside a preamplifier, the difference is that there is literally no interconnect cable. This is important as the interconnect cables have some small capacitance per foot (which becomes a larger factor when higher impedances are involved); this is a non-issue in a dedicated preamp where such capacitances can be 1/100th or less as seen in an interconnect.
The high frequency rolloff thus produced is likely not in the audio passband unless the amplifier has a very high input impedance. The control's series resistance can mess with the Miller Effect of the input gain stage, independently of the rolloffs generated by the cables involved. Generally speaking, this will be more likely somewhere in the middle to upper 3rd of the control range where higher series resistance of the control combined with a high resistance to ground begins to interact with the input capacitance of the input gain stage of the amplifier.
A high frequency rolloff has phase shift effects manifesting as low as 1/10th the cutoff frequency according to the engineering rule of thumb. The cutoff frequency (-3db point) is defined as:
f= 1/ RxCx2Pi where R is resistance in Ohms, C is capacitance in Farads. To get a more meaningful calculation, it is useful to express C in microfarads, thus the resulting quotient 1,000,000 instead, the resulting f frequency is thus in Hz.
A typical cable might have 15pf per foot. With a 3 foot cable and a 100K volume control set halfway across the scale, this results in a rolloff (-3db point) at 70KHz, meaning that phase shift artifact is going all the way down to 7KHz.
At the points of mechanical contact (audio connectors) there is usually some sort of primitive diode issue to overcome due to slightly dissimilar metals used in the connectors. The higher the impedance driving such contacts the more audible their effects become. This will be expressed as an intermodulation, and is not present when the connections are hardwired.
This is why the choice of cable is so important when dealing with a passive control!
Now some of you may have noticed that there is a more complex situation then just the simple example I gave above! In addition to that particular rolloff, we have the interaction of the output coupling cap with the volume control, interconnect cables and amplifier input all in parallel. From the output coupling cap point of view, this is likely to be a negligible value, but from the amplifier input its another story.
The input capacitance of an amplifier is in parallel with the input, which is to say it acts as a rolloff factor. An input capacitance of 25pf is not uncommon and can be considerably higher in solid state amplifiers. This value is operating independently of the Miller Effect of the input stage (which also contributes to rolloff). The same formula applies. If you did the math, you will see that the rolloff at the output of the passive control is lower (if set at the same place in the prior example), in this case about 45KHz. Now artifacts are occurring down to 4.5KHz, on top of those going to 7KHz.
This makes things tricky. Obviously keeping the cable capacitance down is important- if the cables were 6 feet instead of 3 feet, the cutoff frequency would be cut in half, putting the phase shift artifacts an octave lower!
If the control is a higher value, the cutoff frequency is the example above is reduced and can actually be **inside** the audio passband! You can easily see that higher value controls are going to introduce a coloration no matter how good the parts are in the control, as the coloration is a phase shift introduced by the math and is incontrovertible.
Practically speaking, this limits all passive controls to no more than 100Kohms unless fidelity as defined in the traditional sense is not a goal.
Now it is a fact that phase shift in a simple signal like a sine wave is inaudible. This is not true if the phase shift covers a band of frequencies- the wider the band, the easier it is to hear. The human ear/brain system uses phase to establish where a sound is coming from, i.e. in contributes to soundstage perception. In addition, phase shift is interpreted by the ear as tonality- a simple 6db/octave rolloff at 50Khz will be heard as a darkness.
Some years back I had this demonstrated to me in spades, where I was trying to find out (for one of my dealers) why an old MFA preamp was so bright in the phono section. It turned out that there was a step in the EQ curve at 50KHz, resulting in a 6db per octave error in a frequency band that is often thought to be inaudible. But phase shift being what it is, the ear was easily able to detect the problem even though our hearing does not go that high!