I have gone both the passive and the active route. In theory, your are correct that a passive should give greater purity and transparency. And I do hear that with a passive, but I also think it comes at a cost.
The music loses some oomph, weight, drive and robustness. It can get bleached and thin and somewhat threadbare compared to an active. Personally, I prefer active cause I think it makes the music sound more real and alive.
And please don't say I got these results because I did the passive thing wrong. I followed all the rules: short low capacitance IC, impedance matching, amp highly sensitive on the input, gutsy source output, etc.
Others may hear differently and prefer differently. I'm sure anyone can hear the active vs. passive differences. Then it just comes down to subjective preference. Just use your own ears and decide for yourself.