Budget first
then build
Usually agree speakers are deciding factor on a lot. On my current system, I got KLH Model 5;s. They sounded great, plus also looked great with my vintage system. Good size, good sound, good looks, good price. Played with them for about a year, then got Klipsch Forte's. This changed everything. My systems was much brighter, bass much deeper. However, it sounded like crap! Going from 89db to 99db sensitivity showed me everything that was wrong.The Klipsch showed me that I had a ton of noise in my system, my preamp has a slight hiss on the right channel.
What did I do? Got all new cables, pulled all components, cleaned everything, put back with all signal on the right, all power on the left, added a power conditioner, My amp is huge, also had to cut out the back of my cabinet to fit it.
In my quest, the speaker first cost me a lot of unintended money, but now sounds wonderful.