(Unless you plan on getting more speakers and more amplifiers to drive them you can get by without)
Using your 2 channel setup with LS50’s:
I would change up the source. Invest in a Oppo universal player/DAC, the BDP 105 and other models have variable volume control so I think you could hook it directly to your Red Dragon or still use the Schiit. Once you have a great source you can build from there and add to it. EXAMPLE: you could add a music server because the Oppo is a DAC already and a dam good one with USB 2.0 audio input it can be easily connected to anything! , the Oppo's sound is so good it is Stereophile rated class A (BDP105) with dual audio outs XLR and RCA connections. Did I mention it up samples (using some of the best video circuitry on the planet) all your DVD’s to 4K! Oppo makes many models for all budgets.
You could also use a PS4pro as a source it is only $400 and has digital (optical) audio outs so u can use a outboard dac then to your system pre amp , It is a great universal player, video/music streamer and with Playstation Vue u can get all ur TV shows as well.
Matt M