I would do more research. If you are looking for very high quality sound, then you will find it with a very high quality active preamp. The idea that a passive preamp will interfer less sound great in theory, but typically that is as far as it goes. Yes, it can be done.
The preamp takes really small signals and boosts it to an optimal output for an amp. The amp is rather unrefined brut as opposed to the amplification in a preamp. For most of us, the preamp is the heart and soul of our systems. It is the component that brings the magic. A great preamp will make the rest of the system sound better… maintaining the sound quality you want with the choices you made before and after. I currently have a spectacular system, well balanced among components (see under my UserID). If I was going to do something to upgrade my system, it would be to replace my Audio Research Reference 6SE with an Audio Research Reference 10 preamp.
There are a number of people that employ passive preamps to good results, but these are definitely in the minority. Active preamps are main stream for a reason… achieving great sound is really hard and using a passive preamp to achieve great sound quality makes it much more difficult… in component choices and the knowledge level required… this is coming from someone with thousands and thousands of hours of pursuing great sound. You don’t want to make the journey harder.
But if this excites you. Go for it. I am sure there are folks here that will be happy to help guide you.