I am an engineer. I can appreciate some of the subjective comments about anti-skate and how it affects the sound, but I want to clarify a few things here with some facts.
1) Skating force is generated by the friction of the stylus on the vinyl. This friction creates a moment (torque) on the tonearm. This moment is the distance of the friction force from the pivot point of the tonearm. This statement applies to most, but not all tonearm designs. If your cartridge is turned so that the stylus is perpendicular to the tangent of the record groove, then this statement applies.
2) Skating force is constant regardless of the tonearm position, ie. beginning or end of the record. The force can vary due to the groove modulation, but this change is a small fraction of the overall friction force. There is a youtube video demonstrating how skating force is constant with a blank disc. Excellent video.
3) Yes, the anti-skating force on the tonearm produces a moment (torque) on the cantilever, but so does the VTF (Vertical Tracking Force). Ideally, these loads should be applied directly to the stylus, but they cannot. But cartridge and Tonearm manufacturers do not design their products in isolation. So the cartridge manufacturers build a suspension system into the cantilever. This suspension system is preloaded. That means they have already built in a counter load to the VTF and anti-skating forces that will be applied by the tonearm. If you look closely at your phono cartridge while it is cued up- you might be able to see the cantilever sitting down and slightly to the right against the frame or body of the cartridge. But when cued down onto the vinyl the cantilever should be nearly centered inside that framework. At least that is what I see on my cartridge. That is why the cartridge manufacturer gives a recommended VTF range. They have built a preload into the suspension within that VTF and anti-skating force range. If you apply more or less VTF/Anti-skate, then the cantilever may not be in the optimum position.
Without anti-skate applied the left side of the groove is countering all of the skating force. This will not just possibly make the two channels sound different, but it is going to accelerate wear on one side of the stylus. So why do some phono cartridges sound better on some rigs without AS (anti-skate) applied? I'm not sure, but I speculate a couple of ideas, some of which were already mentioned. Perhaps the pivot bearing has some friction. This friction, if high enough could not just counter the skating forces but also cause the right side groove to have to move the stylus. Another possibility is that the tt is not level. Make sure your tt is perfectly level. If it is a suspended tt like mine, then you have to make sure the platter/tonearm is level, not just the base. The last possibility that I can think of why the cartridge might sound better without AS is tonearm geometry set-up. We must use our protractors to set the HTA and VTA in a static situation. That means that while the cantilever is loaded in the vertical direction (VTF) it is not loaded in the horizontal direction. That slight difference could be audible to some. Perhaps some tweaking is required to compensate. When I am setting up my tonearm position, I pull on the platter to load the stylus in the horizontal direction as I position the tonearm.
I apologize if I seem brusque. I think I am correct, but my feelings won't be hurt if someone finds an error or errors in what I have written. It is all about learning.