Would it be an appreciable difference in sound?

Maybe. But it can go either way. You may end up liking the Rotel better.

For a "HUGE" improvement, I think you need to move up to the next level of CDPs. These are typically CD/SACD players or CD ONLY and don't include DVD playback.
The higher up the chain, CDPs will (sometimes) have a better DAC, and most certainly a better master clock to prevent jitter. Lack of jitter control is what causes that "glare" or harshness from a CD.

For example, the Musical Fidelity M6CD would fall into this category, and the higher end Marantz. If I was to put a price on this level of CD, it would be $2000+. Of course there are some exceptions.

It's a gamble. If the cd player you add matches well with the gear you already have, you'll be able to hear the true potential of the player. If it's a poor match, you won't hear the true potential of the player. Beneficial matching is huge in this hobby.