I'm sure your head is already spinning faster than a 45 rpm Mo-Fi reissue, but I'll throw in a couple other ideas you might want to keep in mind. These are observations based on years in the hobby. I'd like to consider them common sense, but sense isn't always that common, and I'm sure many would debate these points.
First, simplicity should be your friend when you're just starting out. Unless you have large quantities of multiple formats of music, pick a single source and work with that. You can always add additional sources later as budget and interests allow. For me, the ultimate source will always be vinyl, but can be a bit more finicky than, say, streaming (which I don't even do). Simplicity also argues for an integrated amp over separates -- one less box and one less set of interconnects to buy. Yes, you can't stagger your upgrades the way you could with separate pre- and power amps, but that's an issue that's down the road.
Second, buying used can save you a lot of money ... unless it doesn't. By that, I mean that some components are tailor-made for purchasing used: amplifiers, CD players, interconnects, speaker wire. With tube gear, should you go that route, you do need to be aware of tube life and to be conservative you might want to plan for a replacement tube purchase in your budget. Finally, and this is just me, but I wouldn't go with used for a phono cartridge if you go the vinyl route. Far too easy to get something that has been damaged or is on its last legs.
Finally, all things being equal -- aren't. While system balance is vitally important and everything matters, I have always felt that the "first among equals" in any audio system are the components that turn mechanical energy into electrical energy and vice versa. So I will allocate a big more than an equal share of my budget for phono cartridge and for speakers. I've found that approach has served me well, and you might too.
Happy listening.