If the stylus is not riding "straight", as it were, the cantilever itself is bent or the assembly to which it affixed is out of alignment. I have had experience with this on two occasions. Once associated with the former condition, a long time ago with a Grado, once more recently, associated with the latter, with a Dynavector. In either case, there is a problem and, hopefully, you can seek remedy from your dealer. If you bought it new, a reputable dealer should assist you in obtaining a replacement.