Perhaps a passive is, well, “too passive” to extract all of that information (on the source CD etc). This requires an “additive” (accurate gain) approach. Our stereo systems must be “additive” to even play a single note through a speaker. A passive may simply leave these higher order musical realities out – they may be subtractive.
Grannyring
Passives don't extract anything. They just pass an attenuated signal. Your source is extracting the music off the medium.
Systems do have to be additive to play music. There should be enough drive from a 2V source and enough gain in your amps to accomplish this. Anything more is excess IMO.
The only way a passive can be subtractive is if there is an impedance mismatch. If you are missing depth, body, and dimension with a passive well...