I have found there to be a huge range of opinion on the importance of a CD player - everything from "they all sound the same" to "they're the most important component". My own personal experience was similar to the ones you have described - I experimented with different setups and compared without noting much (if any difference). Everything from a decent Sony player, to a DVD player, to a DVD player as a transport into a nice pre/pro used as a DAC, to a transport into a decent DAC, etc. etc. I usually look for changes that are clearly differentiable and it's not subtle. I like playing around, but if something would take days to really appreciate, I guess my patience runs out.
I would add that my cable thoughts are between yours and many of the others here - I use cable that all my non-audiophile friends can't imagine springing for, but wouldn't make it into cable afficionado clubs around here.
Anyway, after several years of being unconvinced but interested and open to being convinced, I recently bought a Muse model 9 DVD player that is also highly regarded as a CD source. It was instantly more enjoyable. Two particular things characterized the differences for me. First, the ability to resolve all details on the CD is decidedly better. The most obvious case is hearing instruments on familiar recordings that I've never heard before, but there are plenty of subtler (but obvious) differences in hearing how an instrument is being played, hearing the singer's voice in all it's layers, etc.
The second major difference is harder to describe without sounding vague, but there is no question that the whole sound is smoother without being any more laid back. One aspect of this is that the speakers (I have Dynaudio Contour 3.0's) "disappear" to a much greater degree. But the alto sax (love that instrument!) that sounded convincing before now has a character that is just more "right" (Sorry, I warned you that it would sound vague).
I wouldn't even try to describe this if I hadn't tried at least 10 different renditions prior to this that hadn't made a bit of difference, IMO. My system has gotten better over time, so maybe the resolution of the overall system is bringing it out this time, as well as the player. I don't know. All I know is I'm not going back.
BTW, I think the suggestion to go to a high-end store and audition the difference on a reference system is a very good one. I recently did this with amps and it was really amazing how much just changing the amp (up in the same manufacturer's line) improved the sound. Swap back and forth between a decent and a reference CD player on a reference system. If you can't hear a difference, then by all means stick with the less $$$ approach and don't worry about it. If you do hear a difference, that's probably good news too, but then you'll have to go figure out how to achieve the same difference in your own system. -Kirk