Which is the most important part of a stereo system?

 So I’m definitely not in the audiophile major leagues like most of you guys, meaning I don’t now or probably will not ever have 10k+ to spend on my system. However I’m definitely an audiophile and used to build Martin Logan speakers in the 80s when I was at KU in Lawrence Kansas. I definitely got the bug listening to krell, audio research, and Macintosh driving reference Martin Logan systems. That said,  i’ve been very lucky to piece together two amazing systems on the cheap. I’m driving 1970s klipsch cornwall ones, the verticals, with a Chinese Doge  tube amp at 60 W controlled by an old rotel RC 995 preamp and a crappy JVC CD player from the 90s.
 I was lucky to get an 1988 Sota Sapphire turntable for vinyl for a song from a generous guy moving out of state recently. 
 For streaming i use the discontinued chrome audio, and or a dragonfly 

 I’m sold on klipsch horn loaded speakers for their durability, efficiency and the ability to get an amazing premium sound for reasonable dollars on the whole. 
I have quite a few pairs of klipsch (la scals,cornerhorns,hereseys,) and fix them up and resell them to finance the hobby
   So to conclude I maybe have 2000 bucks into the system I described and my opinion is definitely every component means something to the whole stream. I will say that you can put a lot of subpar amps into Cornwalls or  La Scala‘s and get a pretty good amazing sound, at the very least  A lot of sound pressure with low watts as a source. 
 I do know that putting a pass labs, or McIntosh amplifier into most klipsch speakers is  going to give you  top of the line sound , but 2500 to $6000 for an amp at least for me isn’t in the cards.  I haven’t even started adding subwoofers to my systems but know that would be my next move to complete the systems.
 Again I think I’m like the majority of folks that really want the audioNirvana sound but really just don’t have the income to achieve it. But if you’re patient like me and find some good deals you can put together an amazing sound system for a budget that is possible/ realistic for a large group of people.  I’ve played a lot with room treatments and placement and I think in the end that helps quite a bit. 
   I would agree with a couple of the posts that really decent and in my opinion high efficiency  speakers would be first, then the clean amp, with the sources backing up the list. 

What @almarg said. I suggest not spending disproportionately on anything that (1) wears out, or (2) has low benefit per unit cost, like cables.

Deschutes, why not DIY your electronics? I would say that the main thing is to stay away from teflon insulation, because when teflon heats up past about 450F, it begins to decompose into some really evil gasses. And unless you have a fume hood or equivalent protection, well ...

Lots of good advice here. Millercarbon probably offers the best. For the last several decades I've been through many different audio setups and equipment have learned that one single component or cable doesn't make or break the sound that you want (unless that component is defective).
One example.....I once had a pair of speakers that I really liked but the amp driving them was old and sub par by many respects. So, I bought a really good and more costly amp. The sound wasn't nearly as good (to me) as that from the old amp. SYNERGY is the key.
One single upgrade may improve your sound but everything must work together as a "team". If you have the chance to audition a component in your system and in YOUR room that is the perfect scenario. But that can be a daunting task. There are dealers out there that offer that.
Good luck to you. Remember......good sound is what YOU think it is.