A stupid question for which there's no sensible answer.

Most people have never listen to their system 

How true, but how many times i said "that's it" and how many more i found the next sonic upgrade step within the same system. New, old, oldest, humble, expensive, it does not matter.
Know your system, your budget, and make the changes that suit your preference, from fine tuning adjustments to even adding a new component.
Any change though should not be focused on one thing and others ignored.

G

@hilde45, I have heard $200,000 systems sound bad in great rooms.
It is a game of leapfrog. We Identify the weakest link in the system and improve on it. Sometimes there is a specific problem we want to solve. I always wanted a little better high frequency dispersion than what I was getting from Acoustat 2+2s. Everywhere in the room except dead ahead of the speaker had high end roll off. In the listening position they sound awesome. Could not ask for better. The solution was getting a pair of Sound Labs (still awaiting shipment) By design they will disperse better for sure and they should sound as good. The operative word there is should. I have an Alpha USB to get music off a computer. It only has one output (at a time). I need to send signal in two directions so I purchased the M2Tech trio. They are supposed to arrive today. By one account it is a step down. We shall see. I think it has a better clock. But hey, how many of you own Italian Hi Fi gear? Should be an experience. 
The weakest link varies from system to system. It would usually be best to go after it first since most of us can only do one thing at a time. At some point moving up becomes prohibitive or silly. Nobody has to spend more than $150,000 on a system. IMHO spending more is either a matter of ego or lack of knowledge. I had very wealthy clients that gave me carte blanche. Boy was that fun. It was not just getting them equipment by the way. I installed it all in custom cabinets working with the cabinet makers. Back then you only wanted to see the face plates, nothing else. This was down in Miami. Many of the houses built by the water were up on pylons. There was usually a 3 foot crawl space under the house. I would drill holes in the floor under the cabinets and by the speakers running wire under the house then calk the holes. I would frequently come out from under the house looking like a tar baby. Some people wanted speakers all over the place including around the pool.  
My favorite installation was in an antique armoire. I had to do it without so much as a scratch.
Sorry for getting off topic! Run on thinking. Maybe I'm getting manic again.

Mijostyn: I never heard of Alpha USB, or any other devices serving that purpose. I can't afford one, but I wonder if a cheaper iteration would make a difference as I'm streaming directly from a Mac to a DAC. I gather that the sound degrades coming out of standard USB connections?

1. Speakers, flatter the response and wide dispersion the better
2.Room
3.integrate 1 and 2 with proper treatment and PEQ. 
4. Amp, preamp, source or better yet get active speakers 
5. Dac for digital. Cables are irrelevant as long as they work. USB is fine as long as your DAC is of competent build. 

You say you're going from computer to DAC if you can hear noise  caused by grounding issues or if the computer interferes with playback it might be worth getting an inexpensive streamer.

1. The room (50%)
2. The setup (20%)
3. The speakers (20%)
4. All the rest (10%)