What should I upgrade first on my budget system?

I think your speakers are probably the best component in your system. Typically you start there. But in your case I think you may wait until last. 

I always try and get a big improvement with each new component. 
 

I try and build all components to a new and higher level. Never investing less than double… hopefully more. 

I would probably look for a preamp first… it is central and is so important of setting the tone. Then amp. But one could argue you may get more bang for the buck with the amp first (pun intended). 
 

Next a DAC… which will elevate your digital play… at this point you may use a laptop or other source as a streamer until you can afford a stand alone streamer. Streaming today can easily exceed your CD player in sound quality and replace buying CDs, allowing nearly infinite music for $14.99 per month (Qobuz). 
 

Then you would be ready to upgrade your speakers.

I would look for used audiophile equipment less than say 15 years old. Especially for the preamp and amp. 

@coryb are there aspects of the system that you do not like? Or things that you feel it should better?

(that is where I would start.)

Usually a highpass filter to your amp/speakers… and adding a sub to fill in the bottom would often free up the amp and speakers.

but… that assumes your amp and speakers are over taxed with low notes, or that you want more SPL in the low registers.

So I am not sure there is a rational reason to upgrade without some goal or thing to improve… and maybe just going slow is best?

(And enjoy what it does for now)

 Start from scratch, asking for advice on an audio forum.

@carlsbad +1! Speakers offer the best-bang-for-the-buck! I can only recommend what I use: DCM Time Windows and Ohm Sound Cylinders. Two great classic designs that can be found at affordable prices. Both are competitive with today's four-figure speakers! 

The rest of the OP's system is fine! Depending upon how much he has invested in LP's a cartridge upgrade would be good. A Hana SL plus an SUT is an easy recommendation. Keep the PS Audio preamp and the Will Vincent Baldwin tube amp. Both units offer excellent sound quality!