Well First off, PVC is not a carcinogenic.
Vinyl chloride is.a carcinogen. This is a GAS used in the production of PVC.
PVC is listed as an approved material for potable water. In solid form it's pretty inert.
But you bring up an interesting topic. The real question is how hot does it get at the point of contact where stylus meets record surface and what can potentially be emitted .