Fastest way to correct hifi system?

You're a better man than me if you can build a system "you're satisfied with long term". My wife once asked me "doesn't it sound good enough". Made me laugh. If you buy high quality components to begin with though, you can tweek them (without replacing them) & get great improvements without spending a bundle.   

"minorl"  Has given you some very "sound" advice.  I may throw in my 2 cents worth.  Many will emphasize loudspeakers over everything else.
It is a starting point.  I have been completely surprised at how electronics (amplifiers & preamplifiers or an integrated) can make a loudspeaker that you may not have ever appreciated, sound like you just upgraded it or even sounds completely different.  Please remember that amplifiers that essentially measure the same can sound completely different. Your ears have to make the call.
A dealer will only sell you what they carry and has a bias towards their "stuff" and may even make derogatory remarks about components that you have found to be very pleasing. 
These are important choices that you really want to get right the first time and I am not sure that there is a fast route to achieving this.
Good luck and happy hunting!

1) Invest in a good set of headphones. They will help you isolate the effects of components from speaker cables and listening space.

2) Take your time and savor the journey as much as you can. By the end, you will have a much better idea of what you like and will have developed much better listening skills. It is a joyful process.

3) Don't blow your budget on expensive cables until you have proven to yourself that YOU can hear a difference. If spending $500 on speaker wires is a better way to invest your money than getting better speakers, something is wrong.

4) Decide what media is the alpha dog in your collection. If your favorite music is all on CDs in your library, get a great CD/universal player first and hold off on a turntable & cartridge, or vice versa. Or if funds are limited at this time and you're true love is vinyl. Get a decent CD player and and hold off on the analog front end until you are thoroughly familiar with the rest of you system and have saved enough to avoid compromising your choice of turntable/arm/cartridge/phono pre and tonearm cable - the one cable that makes a big difference. IMHO

5) carefully identify 3-5 really good CDs or LPs that test a system. Well recorded with wide, deep soundstage, the vocals that really move you, the best sax solo you've ever heard, your favorite bass line, the most subtle details, etc. Use them as a uniform set of tests for all listening decisions.

All this is simpler than it sounds.

Best of luck and enjoy the journey.

I wouldn't  say this is the fastest way but the better way for better gains and satisfaction: focus on acoustics of the room.  period.   Too many audiophiles tweak their systems with expensive cables, power conditioners, and even components and speakers.  The biggest element in any system is the listening environment.  One can spend far far less on acoustical treatment that would increase the sound quality of a room exponentially more than upgrading ANY component in their system.   But it doesn't come with Bragging rights...just intelligence.