They will peel off, you just have to get your fingernails under them from the inside and keep plugging. Once you are able to one small part of them up a little bit the rest of them follows easily. They really don't hurt anything by their additional weight, so don't worry about that, but when I was putting some of my CDs into my music server they were sometimes coming loose, which messed things up. I also found a disadvantage to them in that they could get stuck in a car CD player. So I'm removing them as I bring the older CDs down to be burned, it takes a little work but most of them come off fairly easily once you get the hang of it.