The good news is you can get into some really good sounding equipment if carefully chosen.
A really good Phonostage is critical. I would look for something like a used Audio Research PH8 phono stage. This will firmly support a good quality TT from $4K to $30K.
I would look at VPI with a very good cartridge… I think we would need to know your tastes and other equipment to get very specific on that.
if you could spring for a bit more I think you could really get into a really amazing system… say ~$5K TT and $2.5K cartridge + the ARC PH8 (this is a really sweet spot… I owned one for years… I now have a ARC Ref 3… which is better… but at also double the cost).