That's certainly true, generally speaking, but first it is not always easy to determine the weakest link, and then by changing any component your change the overall balance, so you might upgrade but not get better sound or improve some aspects of the sound and make others worse. This is a form of art, I would say.
In my case, I think I know that the weakest links are cartridge and speakers, though they are not weak links, so I have no need to upgrade, only a moderate wish.