For me, among many upgrades done over 6 years, the biggest difference was from Yaqin MC10L (Chinese 55WPC int tube amp) to Linn SS power amp/Rogue tube pre amp -- $550 vs $4800 if new.
The second, from B&W 603 S3 to Tyler Taylo 7U -- $1000 vs $3500 if new.
Third, from $100~$250 MM cartridges to AT150mlx.
Fourth, from NAD C541i to Arcam CD23 -- $500 vs $2000 if new.
... including speaker wires, interconnects, power cable, turntable upgrades, ...
As seen, the CD player upgrade was not as big improvement as with others mentioned, but still big enough to readily notice, and bigger than many other upgrades I've done.
Yes, I gotta dig in CDs to enjoy music again.