P.S. Do you guys have any recommendations for record cleaning and ways to reduce static? I've already got a humidifer running...
Back to your original question...
Go get yourself a pint of Disc Doctor Miracle Record cleaner ($25.00) and Size A wet scrub brushes ($42.00). This will take care of you until you decided you want to dump a bunch more money into this hobby.
Yes, you'll hear arguments for record cleaning machines, various solutions, approaches and techniques, the necessity of demagnitizers, isolation platforms, mats, pebbles in a jar, etc. that you *have* to have to get the most out of vinyl. For now, indulge for amusement in between spinnin' your freshly-scrubed vinyl. Don't be discouraged by those who insist that you need to spend a ton to enjoy this aspect of the hobby. There is a time, place, and price for everything.