Why do mass marketed CD's sound so crappy?

I posted awhile ago here asking opinions regarding the poor sound quality of Coldplay's "A Rush of Blood to the Head" CD. Now I want to ask the same question of U2's latest (which is great, btw). I also find Sheryl Crow's CD's to sound underwhelming and dissapointing. Besides that fact that I love her music. What gives? Are the artists clueless? Don't they hear what their releases sound like? Are the record companies deliberately turning out crappy sounding CD's to please the masses that listen primarily on Ipods and walkman's? Man, it makes it real tough to enjoy music I really love to listen to when it sounds so damn bad.

The first track on U2's newest, "Vertigo" really rocks out, but it sounds boomy and muddled. I wanted to turn this up real loud, but it just sounded awful. I'm bummed.
The dynamics are compressed and the overall volume
is increased to be LOUDER on the radio. This takes a heavy toll on the finer aspects of the sound quality.
Upside is my wife has practically stopped listening to country western music as the production quality of most artists is abysmal. She's moved closer to blues, jazz and celtic.
I think they're just clueless, both the artists, producers, record companies, the recording and mastering technicians, etc. Give it to have that "loud," "punchy," "fat" sound to have it stick out on radio play (or in boom boxes). I don't think a lot of people realize what is possible with high fidelity systems. I know I didn't prior to 2001. I thought more expensive systems were just about BASS AND POWER. I think people just don't know what should be possible. they think "loud, punchy, fat" (and compressed, lifeless, and dull) is the best thing. And it isn't.
Birdwizard is right on. In our audio society we went to a recording studio where we heard a local artist do up a first shot at a song he'd just composed, and got to see the various tricks they could perform with the pro tools software in the studio. A few months later the guys from the studio came to one of our meetings at my home with a CD on which was the original cut of that song we heard, the rough cut with the full rock band in the studio and the version they were presenting to the record companies to sell the band. The rough cut sounded great--dynamic as hell, clean, ideal for an audiophile's system. The cut the record company was getting was compressed dynamically and nowhere near as good sounding as the rough cut. But it is what is needed to make the song sound good on a radio station (tough to hear the really quiet parts over the road and wind noise in a car) and to sell the record and the artist to the big labels. Don't necessarily blame the artist, it's not their call for the most part.