Online vinyl?

also
www.amusicdirect.com (the "a" is not a misprint) has turntable packages and LPs and CDs, excellent catalog.
I would recommend VPI HW 19 MkIII (retail 1250) + Origin Live modified Rega RB-250 arm (about 600) + Dynavector 10X4 Gold (450) + EAR 834P tubed phono amp (900). That's in the ballpark price wise.
I haven't listened to this but based on some hunting around, I'm sure this would sound fantastic. You might have to put it together yourself, but give musicdirect or needledoctor a call and see if they can put together a package.
there are lots of other nice choices. In general stay away from the solid state phono amps, you'll be happier with the sound of tubes in the long run (and may want to convert to tubes for the rest of your systemas well!)

My setup is J.A. Michell Gyro SE $1500, Rega RB300 $425, Benz Micro M.9 $1000, AR PH3 hybrid phono $1000(used) + a collection of 600 records 75%rock(progressive, underground, classic), 20%Jazz, 5% classical that I've bought on different websites, garage sales, countries and stores averaging $4 per piece.

¿Tubes for phono? No way! There's too many good transistors around (Klyne Audio Arts for example).

Might I add that a good record cleaner such as the VPI 16.5 (or 17) or one of the others would be a good addition. There are lots of excellent records out there if you ask around at work, neighborhood, friends, etc. (Most will assume you're nuts and practically give them to you in sympathy.) Having a cleaning machine at the ready can turn a very good album into a great album, sonically speaking. Even new records can benefit from a good cleaning!

As to your question on playback equipment, tables seem to break into basically two schools of design- suspended (i.e. Linn Sondeck) and unsuspended (i.e. Rega). Seems that most folks prefer unsuspended, at least from my casual observation. The "sound" is different (tighter, more focussed) and you don't have to fool around adjusting the #$%!& suspension. I happen to have a Well-Tempered Classic (unsuspended type)with a Benz L0.4 and highly recommend it, but there are a number of very good picks out there. (Had a Rega P3 prior to that and the WT was a big improvement.) The phono pre-amp is "gold card" (SS) option on the Audible Illusions M3A pre, which is a marked improvement over the already-good "standard" phono section. (Would've liked to have gone with a tube phono pre, but had to hold cost somewhere!)

As far as tonearms, I second Tacs' suggestion on the Origin-modified RB-250. It can be had for much less than $600 at GCAUDIO dot COM.

Psychoanimal- These boys are in a different league. Don't worry, though- they know what they're talking about!