There are so many variables (including listening preferences) that, as others here have pointed out, there really is no one answer.
In addition to having enough voltage to drive the amplifier, impedance matching is also important with a low output impedance (from the passive device) feeding a high input impedance (to the amplifier) being desirable. The length of the IC cable between the preamp or passive device and the amplifier(s) also affects this.
You cannot lump everything that has gain into one "preamp" basket because there are many different designs. When folks say that preamps are no longer needed, I believe they are mostly referring to the high gain devices that were common 30 years ago.
The optimal set-up for a passive preamp is between a source with sufficient voltage output to drive the amplifier, and an amplifier with sufficiently high input impedance to not be negatively affected by the output impedance of the passive device, and finally for the connecting IC cable to be sufficiently short to not have an effect.
Even when these things seem appropriate, there are many who still prefer sending their signal through an active "preamp."
The issue is further complicated in that all "passive" devices are not the same. Some passives are simply a volume control, including a pot or perhaps a discrete resistor type control, and others use transformers, autoformers or even LDRs (light dependent resistors). The resistor based volume controls seem to be most affected by impedance issues. I own two resistor-based passives, a Goldpoint single input unit and Endler attenuators which connect directly to the amp so taking the length of the IC cable out of the equation. Both use discrete resistors and both sound very good but need careful impedance matching between the source and the amp. I have also had the Acoustic Imagery JaySho here for an audition (this is the same design as Chapman's Bent Audio TAP unit, but repackaged) and while it may open up a wider range of equipment that will work well, IMO there are sonic trade-offs so I wouldn't necessarily say everyone would like the TAP unit better.
George posted a portion of a Nelson Pass quote previously in this thread that was taken from Pass' discussion of his DIY "buffered" preamp. A "buffered" preamp is generally considered to be a no-gain device with an active electronic circuit that reduces the output impedance. Here is the other half of Pass' quote;
Is impedance matching an issue? Passive volume controls do have to make a trade-off between input impedance and output impedance. If the input impedance is high, making the input to the volume control easy for the source to drive, then the output impedance is also high, possibly creating difficulty with the input impedance of the power amplifier. And vice versa: If your amplifier prefers low source impedance, then your signal source might have to look at low impedance in the volume control.This suggests the possibility of using a high quality buffer in conjunction with a volume control. A buffer is still an active circuit using tubes or transistors, but it has no voltage gain – it only interposes itself to make a low impedance into a high impedance, or vice versa.
If you put a buffer in front of a volume control, the control’s low impedance looks like high impedance. If you put a buffer after a volume control, it makes the output impedance much lower. You can put buffers before and after a volume control if you want.
The thing here is to try to make a buffer that is very neutral. Given the simple task, it’s pretty easy to construct simple buffers with very low distortion and noise and very wide bandwidth, all without negative feedback.
Link: https://www.passdiy.com/project/preamplifiers/b1-buffer-preamp
Other manufacturers have also found the no-gain to low-gain approach to work well, such as Steve McCormack who originally offered no-gain or +6dB options on his highly regarded VRE-1 preamp, which is basically a very high quality buffered circuit. The +6dB option is standard on his current version of the VRE-1 with this relatively low-level of gain resulting from high quality output transformers he uses, and not the active circuitry. Steve has found the power supply to be very important to the quality of sound of the buffered preamps he has designed.
My personal preamp is a no-gain, buffered unit that was custom built by Steve McCormack and is very similar to his early VRE-1 preamp. It works well with every amplifier I have used and sounds better than the over-20 highly regarded preamps that have paraded through my system over the years. Below is an excerpt from a 1995 interview with Steve where he was discussing his thoughts at the time on passives and buffering. A link to the full article follows.
The purpose of taking the time for this post is to provide information showing why I believe there is no one answer or product that is going to work well for everyone and to respond to the OP as to why simply taking things out of the signal path is not always a recipe for success.The sonic performance of the finished product, called the Line Drive, fulfilled all of the original design goals. It was characterized by the exceptional transparency and natural musicality which are now recognized as classic passive features when the preamplifier is properly designed and manufactured, yet rarely heard from other types of components.
With further experimentation, I discovered that the very best performance of my design came from running it with a high input impedance (20 kOhm or higher), high gain power amplifier. This too is characteristic of all passive units. The amplifier's high input impedance prevents loading effects while the high gain assures ample capability to drive loudspeakers to an appropriate volume level. Only extremely low loudspeaker sensitivity combined with low gain amplifiers present a problem for attaining high volume levels. Again, this combination of factors is rarely present in the real world.
While investigating early reports of "insufficient" volume, I found that people simply were not turning the volume control far enough to achieve the levels they wanted. Following the lessons they had learned about not turning up the volume of active preamplifiers too much as a precaution against system damage, these individuals were setting their controls too low. Since passive line-level preamplifiers have unity gain (1X), there is no reason not to turn the volume all the way up if that is required.
Please understand that I have nothing against active circuits, and I assume the same is true of other creators of passive designs. I build active units for applications where they are appropriate, such as phono preamplification. However, I prefer the passive path in some other applications because of the exceptional transparency and natural musicality possible through this approach.
During the latter years of the original Line Drive's seven year tenure, I worked with design refinements in preparation for releasing its successor. These experiments confirmed most of my earlier conclusions, but they also led me to appreciate the advantages of offering a buffered output in addition to the passive output. I had resisted that option for a while because the buffer circuits I knew had a negative effect on sonic performance. It was during the development of the Active Line Drive ALD-1 preamplifier that I designed a special buffer circuit that was based on a simple complimentary FET pair, but which required a very high quality power supply to make it work properly.
This was the first electronic circuit I had worked with that complimented the passive circuit and did not compromise sound quality. It allows the use of any length of cable between the preamplifier and power amplifier. Adding the buffered circuit brings the opportunity to enjoy the sonic benefits of minimalist preamplification to all systems.