while your strategy yields sonic improvements, a very modest (in terms of cost) DAC upgrade would yield “significant” sonic uptick. There are many choices.
We look at your entire audio chain and the DAC seems to be the weak link.
FYI- in the last ~10yrs+ digital audio has made significant sonic improvements especially in DACs