The Hardest Naturally Occurring Substance on Earth

Yep - You all know from grammar school that is the diamond, which incidentally is what is used to make the stylus of our turntable cartridges.  If it is so hard, and it is going up against some fairly soft vinyl, why do we worry about poor quality LPs damaging the cartridge or stylus?  Sure, I understand the cantilever, but the actual Stylus?  The old phrase for me is "Does Not Compute".   What are your thoughts and insights?