Short story is, everything matters with PC audio. The cable quality, electrical noise within the PC, PC power supply pollution, yada yada.
MP3s will never sound good, upsampling does nothing for them, and upsampling any file type/quality is not a guaranteed improvement.
USB works well, it's what I use, but often an external signal improvement device (reclocker, etc.) makes a world of difference.
I built a custom PC server, all custom "audiophile" parts. Expensive. Then spent another $2-3K on external signal improvement and it made a big difference. Feeding into a $4k DAC. And there's even more I could do.
So, it never stops, you are at the bottom of the ladder now and there is a lot that could be done to improve.