Hi Nandric,
In a literal sense the arm does not follow the groove, or only as much as the cartridge body. I agree, it's the stylus that tracks/traces, the groove.
Considering different aspects of record player functions, it's usually simpler to reduce to two dimensions like horizontal and vertical, but tracking is in four dimensions in-groove (don't forget time).
Rather than add dimensions with other functions, in reality the other functions also operate within those four dimensions. If you consider the movements of the stylus as being within an imaginary sphere of tracking, the movement of the arm across the record can be seen as a separate function operating in the same dimensions of time/space. The same would go for vertical movement - tracking a warp. Neither of those functions is completely horizontal or vertical.
A record player is a bizarre device. Maybe that's part of why we love them. "They" used to say it's like a bumble bee, by the laws of aerodynamics it should not fly. It turns out they greatly underestimated the speed of the bee's wings flapping.
Regards,