Should I eliminate my preamp?

If you have a solid state source like a CD player, quite often it has a coupling cap at its output

What Ralph mentioned about coupling caps above is not relevant these days, as nearly all solid state sources have been direct-coupled for years, only tubed sources may have this problem as they have a caps on the output.
But below tubed preamps have this problem at the other end of the system with some of today's low input impedance amps.  

All is not good a match with 99% of tube preamps with  low input impedance amps these days, many Class-D's, some First Watt's, some Pass Lab's and many others.

The coupling cap that's needed on the output of 99% of tube preamps can be too small for these amps I mentioned, which can be as low as 10kohm input impedance. 

This is not a good match and will roll off the bass very early, to the point of owners saying this "tube pre has little or no bass".
  
To fix this one can change the cap for a much larger uF one which can be a fitting issue in itself or revert to a not so good sounding bi-polar electrolytic cap which will be small

Cheers George 

Willemj- how do you know what an album is supposed to sound like? Are you telling me by bypassing a preamp you are closer to the master? If you like vinyl these guys will tell you their master is more realistic. 
There are many reasons to go thru a preamp:
1) if your dac/cd player is solid state, you might want to go thru a tube preamp that front end solid state amps
2) you might want to upgrade to an external dac, add a tuner, turntable phono preamp, etc...

Bring home a couple of good preamps and compare them to your direct connect setup

You may think at first that you're hearing a more transparent signal but a preamp properly matches the impedance between stages and provides more muscle (better dynamics) in the long run if it's a fairly good pre which it sounds like you have. You will get more dynamics and you won't be as susceptible to roll-off of high frequencies. The pre-amp is more than just a handy switching device, it provides an easy load for whatever you plug into it and outputs a lower impedance which will make the amplifier happier than if you just plug in a source device.

The first truly high end piece of gear I bought was a pre-amp. I couldn’t believe that change it made in my system. Without changing my midfi CD player or power amp, I was hearing a 3 dimensional sound stage for the first time, and a less brittle, much warmer and rounder sound. It excited me so much I rushed to purchase a high end power amp. It added bass authority and an even rounder more musical sound, but the soundstage magic was still a product of the preamp. I’ve been a preamp believer ever since, and would not consider using only a DAC with a volume control. That’s my limited experience. 

I most certanly prefer my Audio Research LS27 to no preamplifier by far.  In fact if I play my 35 year old or so Stax headphone system thru the record out of mt LS27 instead of the audio source directly in the audio reproduction is extremely better thru the tube LS27.  It is not just a minor improvement in sound quality.  Whatever audio source is used, say even my OPPO 205 as well as 105, the addition of a fairly recent Audio Research tube preamp will be the best choice.  By doing so a quality close to40 year old Stax headphone system can actually compete favorably to some of the recent expensive planer magnetic headphones of today.  For a fraction of the cost.  That is what a true quality tube preampkifier can do for an audio system.  It is simply someting one has to try out for themself.  Recently Paul McGowan of PS Audio said the same on his own current top audio systems.