Assuming that you have a nice system which you like, you might want to consider the following. I save my money for the physics - that is, if and only if the physics is solid and uncontroversial. Then I have been known to spend real money.
Those areas are: capacitors (film and foil are the best), resistors (nude Vishays are best), solid connectors (not necessarily expensive, just solid), clean power, clean stylus, and clean records. And the most cost effective tweak of all: little rectangles of sticky jelly-like substance, which can be stuck on things to control resonance, Moongel, for $10. I use the stuff extensively on my air bearing tonearm.
You may notice some high-mark-up items missing from my list. So is everything controversial. That's how I approach the subject: physics first. YMMV