It really depends on where you are in your system evolution. If a person is just starting out, then money loaded into speakers encourages him/her the most, because where that person is at, that expenditure is usually seen as 1) economically justified and 2)musically justified.
That said, it also depends on the person in the above beginning scenario; if they've had extensive exposure to your more advanced system and wouldn't be satisfied and are more attuned to the deficencies we all know, then spread the money out more, distributing quality up and down the line looking for a balance in quality, a slice of what could come, a slice of what you have.
On more advanced systems, of course, everything becomes important, but the skill of balancing becomes most important. And, particularly, towards spatial and harmonic factors. This usually leads towards tweeking at the front end. With that said, however, simply because advanced systems end up spending a lot of time on TT's doesn't necessarily mean that that is the most "important", just where, at that point, the most increases in quality become most apparent.
When designing a system, and some of you will think I'm crazy, I've found starting with the preamp (never a passive) and working outward is the easiest (less having to turn around and fix a sound problem)and results in the most musical sound, while also leaving the most places to expand musically (space and harmonic considerations). IMHO, the preamp is the fulcrum of a system. Then again, just because it may yield better long-term results, to work out from a pre doesn't necessarily mean it is the most "important".
It is a balance and depends where you are and what you want. Bad speaker choice, however, is terminal.
