One big change that no one has talked about is that since the Redbook format was introduced in the mid-80's, the music market has totally changed. It is now a global market - using globally standardized components - marketed by all sorts of low cost general (as opposed to specialized) retailers.
A $300 system today is in fact a whole lot better then a $300 system was then. It's more compact because nowhere else in the world do people have the luxury of space we enjoy in the US.
Add to that what we have seen in telecomm. Emerging nations are not building phone infrastructure with poles and copper. They are doing it using cellphones and GSM and WiFi. People are communicating using Skype because it makes the most sense.
Convergence of media is well under way - take a look at the soon to be released iPhone and the many musical phones already available. People want to be able to access and share without regard to whether it is a music file, a movie file or a still picture.
All of which is a roundabout way to getting to how people want to buy and store and manipulate music. It is going to be digital because it is cost effective, it is environmentally sound and above all because it is convenient. No one listens to an entire side - and digital is certainly an easier way to make a mix tape then the countless hours we used to waste making cassettes.
Which leads to my final points.
So far, digital formats do not seem to be any harder or easier to pirate then disc based media. Counterfeiting is an enormous problem - from the point of view of the content owners, any thing new will have to have very robust DRM without pissing off the consumer who wants to buy once and use it everywhere.
Next, because this market is ever more about volume, I don't think there will again be the critical mass to establish a new music only physical format. I tend to think that the work will be in the BluRay/HD area - that is to say a more flexible format which can be used for many kinds of media and perhaps with write many capability as well.
Finally, to get back to Schipo's original question. I think the time will come (soon) where you can't give the CDs away - not because they are sonically inferior by audiophool standards but because in their own way they are every bit as cumbersome as vinly was when CDs were introduced.
Hard as it may be to accept, people want to hear what they want to hear, when they want to hear it, where they want to hear it. They want to share it with their friends wherever they are in the world. No physical medium can deliver all that. Just look at how fast the iPod and its brethren replaced the Sony Walkman...