I post this only because I don't find this particular advice anywhere else here.
The phrase "you have to start somewhere" is very true, but it assumes later upgrades. Every audiophile I know has the same disease. Don't overlook that. If you are just starting out, you can save lots of money by doing it very smartly. I guarantee you, when you have bought what you think is the best you can get at that time with the given amount of money, sometime later (and probably shortly thereafter) you will start second guessing your choices, and get the itch to upgrade. After two or three years or so, several total system changes, and two or three times the money you initially spent, you may look at your system and say to yourself, "you know, if I had thought of my current system's components two years ago, I could have bought them with the original 5k and saved a lot of money."
You say in response to the above, "yeah, no kidding, lots of audiophiles do that; it's a process you have to go through." And I agree to a point. But my advice is to try to skip all the components in between your "starter system" and your final (is there ever one?) system. You do that by buying the best "low-hi end" components you can find used for the cheapest amount (i.e., Polk RT35i speakers, creek 4330 integrated, some monster wire, and a 200 dollar CD player, all for less than a grand). THEN, after listening to that for a good 3 months, you have a frame of reference of sound from which to compare, and most importantly a system you can home demo components on and hear a comparison. Even if you go listen to every item at all the local dealers there are far too many variables to change the sound from when you listen to it there to when you get it home. Again, you need a frame of reference to start from.
I wish someone had offered me this tpe of advice when I first began. One other tidbit; if at some point you are nearly satisfied with your system but you feel it is missing something specific (like a subwoofer, or good cables, or whatever), don't settle for something well below the quality of the rest of your system. You will just end up upgrading it anyway soon thereafter. Buy the best you can afford in line with the rest of your system.
Best of all, the whole damned process is a load of fun, which ever way you do it! Good luck. (And btw, back-issues of the Absolute Sound have reviewers' choice systems, to include their budget systems. You can get a lot of good ideas from there).