Which makes the biggest difference?

It depends. The components that really matter are the: Streamer, DAC, Preamp and Amp. All matter. The biggest difference I have experience over the last ten years has been the streamer. I had been thinking it couldn’t matter. I was so wrong (just like every time I think something can’t matter). The reason the difference was so profound is I had very high quality, compatible CD player, DAC, preamp, and amp, and an inexpensive streamer. So introducing a compatible / synergistic streamer was profound… like a turntable it is all the way at the beginning of the signal path. If it is noisy and not high resolving, the problem doesn’t get fixed along the way. So, this is really important.

On the other hand, the preamp is the heart of the system, it’s boost from line level sets the character. At the center of every great system is a really fantastic preamp.

But a really good DAC is required as it crafted the digits into the sound you are going to hear.

You don’t want the rest of the signal chain down.

My experience has led me to notice that matching/synergy of components makes more of a difference than any of the components themselves. 

@ghdprentice says it well..

You don’t want the rest of the signal chain down.

so which is the most important link in a physical chain? hmmmm

No clue what I had intended that sentence to say.

How about: You don’t want that signal to go down to a weak amp or inferior speakers either.

While I can make rationalizations for why each component matters, particularly those at the front, the reality is they all matter. If you have a weak link, improving the weak one can make a huge sonic difference. But when analyzing the problem, it is best to keep in mind the front end and preamp as having additional weight in the analysis.

OP's questions leads us to a game of Six Degrees of Audiogon Thread Separation. Can't talk about amp impact w/o mentioning the speaker in use. Can't rank preamp above or below DAC without discussing analog vs. digital volume controls and relative quality of them for the DAC and preamp in question. 
Please refer to EVERY DARN THREAD EVER POSTED for the complete picture. JK. 
Can I add "the mastering of the recording is most important"? 
No? Fine. It's the DAC. Cheers,
Spencer