Can I clean my records manually?

Yes you must certainly can. Use a Disc Dr. pad, alcohol solution in a spray bottle and flannel rags to dry. It works very well. Lay the record on one flannel rag spray each side and brush well with the pad and then dry with the flannel rag.

Thanx gentlemen, can you give me the ratio of alcohol to water?.

Read the Record Playing Rituals stickied at the top of the page. There are tons of ideas on record cleaning there.

BTW, as you correctly surmised, the biggest risk in playing dirty records is not to your stylus. There will be some additional wear but unless you play very dirty records for very many hours the additional stylus wear is likely to be minimal.

No, the biggest risk is to your vinyl, and this is a serious risk indeed. Before I knew better I did audible damage to brand new (ie, not very dirty) records by playing them without cleaning. It is no exaggeration to say that a single play before cleaning can permanently mar a a vinyl LP. This is 100x truer if the record is used and dirty.

Vacuum cleaning is the way to go and, as Flyingred pointed out, it needn't be expensive to get into. Search the archives here and on VA for dozens of inexpensive DIY RCM's.

Your supposition about dirt being softer than vinyl overlooks one extremely important factor, the extreme heat and pressure generated at the point of stylus contact. During that brief moment, those prodigious stresses soften the vinyl so that it is much more vunerable to damage. Plus, the make up of "dirt" varies. You've got fiber & dander particles as well as harder gritty bits of plaster, sand, and who knows what else floating around. I've looked at the detritus on a records' grooves through scanning electron microscope imagery, it varies greatly in composition and size. These variable particles also account for why a dirty record DOES wear diamond faster than a clean record. It's a given that even though diamond is the hardest substance, it can still be slowly worn away. Any increase in abrasive presence during playback exacerbates wear.

Photon is correct. Do the math on a 2 gm tracking force on a .05 x .07 mm line stylus and you will find that the pressures are enormous.