In general usage, a passive preamp is one that does not have an amplification circuit. I.e., its volume control acts only to reduce the volume, starting from the full input signal from a source. Most sources output at high enough a level that they can drive a system pretty loudly without any amplification at the preamp stage. As a result, all the preamp really needs to do is reduce the level to a loudness that the listener wants to hear. This can be done in a number of ways without requiring a power source inside the preamp, including using a variable resistor in line with the signal or using a transformer.
Of course, there will always be instances when the listener may want to hear something louder (perhaps because his speakers are very inefficient or because of a signal level-diluting impedance mismatch at his amplifier), and this would require that the preamp increases the level of the signal using an amplification stage. Such a preamp would generally be called an "active preamp" because it would have an active circuit that would require a power source, usually wall current or in some instances battery power.
Of course, a phono preamp would have to be active since phono cartridge output levels are usually quite low and their signals are not recorded with a flat frequency distribution and thus require RIAA equalization, but that's a different subject.
Reading further in to your question, I agree that all preamps are passive but only in the sense that they are not intended to drive speakers directly. "Amplifiers" or basic amplifiers are required to drive speakers. (When the preamp is built into an amplifier, such a unit is called an "integrated amplifier.") Your use of the word passive is in a broader sense than the audiofile use of "active" and "passive" when referring to preamps.
