Cleaning Records

I have a rather small record collection made up of about 25% new records, 25% old albums that I've purchased from local shops, and 50% old albums of my dad's that have been sitting in the garage for a good 10 years.
As far as cleaning goes, I guess the obvious part is buying a carbon-fiber brush to dust them off before each play, but I'm lost as to what I should do to REALLY clean them. I only spent about $400 on my turntable, so buying a VPI record cleaner used for $800 doesn't seem right for me right now. Are there any cleaners that do a comprabale job for under $100 if possible, possibly $200. How should I go about cleaning without a cleaning machine? People have talked about washing their records. Does this process actually include holding the record in the sink and pouring deionized water over it? How would I clean it?
If I were to clean it with a cloth, would I move around the record in circles as opposed to moving from the label outward? I've seen a lot of "record-cleaning solutions". How does one use these? Just mix it in water and pour it on the record? Wouldn't it harm the record to actually apply force to it when "scrubbing"?
I found a bottle of D4 that I bought a while ago. Should I use it? And if yes, do I dillute it in water first?
I'm obviously very new to all of this, and I would appreciate any help.