On a grooveless record the POINT of the stylus rides on a flat, unmodulated surface. The contact surfaces don't touch anything.
When you listen to music the SIDES of the stylus ride on a pair of opposed, modulated walls. The point never touches anything.
These are two quite different conditions. Extrapolating from one to the other is fraught with untested assumptions. By analogy, you tested Z-rated tires on a warm, dry track, found them to perform well at a certain psi, then extrapolated that psi to winter tires for driving in snow or off-road tires for driving in mud or sand. Experienced drivers know better.
Skating forces do exist. They can be approximately compensated for with anti-skating. I say "approximately" because on real records the amount of skating force is ever-changing. The question becomes, how much (if any) anti-skating to use for real records? There's no simple or perfect answer and there are several factors to consider. Here are some:
1. Anti-skating may extend the life of a stylus. OTOH, cartridges with modern stylus profiles like the XV-1S tend to wear out their suspensions before their styli. When you have that rebuilt you'll get a new cantilever-stylus anyway. (Actually, the XV-1S tends to have its full monty cantilever snapped off before ANYTHING wears out. Of course playing with an unprotected stick is another issue.)
2. Anti-skating stresses a suspension, so one could argue that it further shortens the most short-lived part of the cartridge. Hah!
3. Anti-skating impairs sonics. You said the sound doesn't change that much, but then rather vaguely said anti-skating makes it "warmer". It changes the sound in my system and I hear very specific things. As with any form of dampening, pre-pressuring the cantilever against the suspension attenuates HF's, slows rise times and reduces amplitudes. This could be called "warmer", I call it dulled and lifeless. Like Audiofeil, I've heard this with the XV-1S and other top level cartridges, including several of my own.
4. My amp and preamp builder, Nick Doshi, states as one of his core design principles, "allow as few gain blocks as possible". If you've never heard one of his units you may not appreciate the significance, but I do. Anti-skating is a gain block. It inhibits cantilever movement in exactly the same way that excessive VTF does.
There's no right or wrong (other than using grooveless records to adjust parameters for playing grooved ones) but FWIW I bought my system to play music and choose to adjust it for optimal sonics. If this slightly reduces some component's lifetime, oh well. An occasional Shiraz or single malt may effect my component lifetime too, that's a tradeoff I'm willing to make. Of course there are risks I won't take, regardless of the excitement, but we'd have to discuss those on another forum. ;-)
Doug
P.S. While experimenting with lower and lower amounts of anti-skating I actually went through a period of tweaking it for each LP. The differences in tracking performance and sonics from very tiny changes were quite audible. I now use none because my main cartridge no longer needs it to track even the toughest passages.
P.P.S. As I no longer use anti-skating, I removed the device from my (TriPlanar) tonearm altogether. This had a small but beneficial effect in lowering the arm's noise floor. The fewer twiddly bits to vibrate and put noise into an arm, the better.