Nothing wrong with dipping your toes into vinyl, and seeing where it goes. You can build a nice collection a little at time.
There are solid choices in the $500 range. Much of the quality boils down the cartridge and the phono stage. I've been running a modest AT-LP120X (modified) with a couple of good carts for 3 years, and have been totally happy. Several of the AT tables come with the AT-VM95E cartridge, which is a nice entry level cart, plus it has a great upgrade path with the ML and SH needles.
The Fluance RT85 offers a decent table with a good Ortofon 2M Blue cart.