The one component that you wish you had not bought

Cardas quadlink cables. When I replaced them with entry level Purist I couldn't believe hearing such an improvement in clarity dynamics and balance.
Also, entry level old Nakamichi cassette deck. Quickly replaced it with 682ZX.
And Parasound belt drive cd player that had CEC transport. Immediately replaced it with CEC with the same transport but different converter. What a difference.

1+ for mofimadness :-)

Some massive KR Audio amp, I forget the model number, but it weighed like a 100 lbs and was wide and very deep in terms of dimensions and almost impossible to move. Plus it used very expensive KR Audio tubes, their 300B variant the 842vhd and of course one burned out very quickly. Sold it as soon as I could. Sounded fine when it was running but nothing special to my ears.

Placette Audio Passive.  I am a passive enthusiast, and had been on the lookout for an exceptional passive line stage with the convenience of a remote control.  After reading all of the reviews, it seemed that my long sought-after goal had been realized.  Unfortunately, that was not the case - even a $500 Audio Synthesis Passion far outperformed the Placette.

Miltek Aurora cartridge. Warm and woody. No detail.
Solid silver cables from a great guy. They just weren't as good as cheap 14 gauge mains wire.
CV4004 tubes. Dull as bedamned.