Will I notice a difference?

My bet is YES...you will surely hear an improvement, and I have never even heard the Orchid; but I say so after owning 2 non oversampling DACs, which I consider to be far more natural sounding than other good DACs I owned previously. It's hard It's hard to explain the difference until you hear it, but when you do hear it, it's obvious.

I've tried sub $1000 dacs like Topping D50 and heard no difference at all.

Well there you go then.

What is the rest of your equipment? What do you think is lacking in your sound?I just went through a similar exercise, Topping D50, Sabaj D5, Schiit Bifrost multibit (I already owned), ADI-2 dac.
I kept the ADI.
The differences in sound were marginal. I guess if you were to break the 2k barrier, you might notice a difference.
If you’re staying in the under 1k , any decently designed dac will do a good job at giving you an enjoyable experience. Most of the difference that I found came down to the output stage.

Where else could you put that money, that would make your listening more enjoyable?
just some thoughts.

good luck 

mdr

I have the Orchid, and it is not my most expensive DAC but I think it's my favorite. I've never heard your CD player so I don't know if it will be better or if you will like it better but if you like neutral detailed "analog" sounding digital then give the Orchid a try.

There are other things in your system that will make more of a difference than an external DAC but if you don't like digital sounding sources then it's worth making the investment in my opinion.