VTA my compromise in setting

Let’s put this into perspective.  With a 9 inch tonearm, a 1mm change in record thickness represents a 0.25° change in stylus rake angle.  In addition, some stylus profiles are less sensitive to small changes in SRA.   My suggestion, set the optimal sra based on the most common thickness record in your collection. Also, please note that once you change the sra, all other cartridge parameters also change.  Set it once and forget.  
 

Thank You @testpilot! If you can hear the difference between the various record weights you need to have a long discussion with your shrink. It is more likely that the variation between record Lathes is greater. Set Your SRA at 92 degrees on a 180 gm record. The stylus has to be at it's proper VTF and loaded (on the record). The very best cartridges are constructed very accurately and all you need to set both SRA and azimuth is a Wally Reference tool. This would be cartridges in the $6000 plus category, but it never hurts to check. The less expensive the cartridge the more likely the stylus is to be mounted askew thus the measurement has to be done on the record. Draw the 92 degree angle on a 3 X 3" piece of white cardboard. Use the finest pen or pencil you can find. Prop it up on the record directly behind the stylus so that your line intersects the stylus. With proper lighting and a hand held magnifier adjust the VTA  so that the center axis of the stylus is collinear with the line. If you have a Gyger S stylus or an Ortofon Replicant 100 stylus this method does not work. These styluses are not symmetrical and the measurement has to be taken off the rear surface of the stylus.  

I recently mounted an Audio Technica VM95ML and the stylus was tilted forward 3 degrees (5 degrees off). The tonearm was as high as it would go! I have not looked at any similarly priced Ortofons but I'd bet you dollars to donuts you would find the same sort of thing. With these cartridges the labor to do QC is more expensive than the cartridge!

Thanks for your responses 

My cartridge is a Soundsmith Carmen mkII the TT is a Merrill Heirloom and the tonearm is the Classic Six that does not have the micrometer .

When using an Azimuth gauge/block to check the tonearm and cartridge parallel orientation to the record the difference between a 140gr and 180gr is very small , I would say that it would be less than 1mm , but this has caused me to think about trying a 1mm mat and setting the arm for a 140gr record and using the mat for heaverier records bring the two just a little closer .  Will a 1mm mat withstand the use of being placed on/off frequently ? If you watch Peter Ledermann's  video on SRA I am probably close enough with leaving my setting using a 160gr record .

P.S. correction

Setting the VTA for a 180gr record and using the mat for the lower weight records. 

@vair68robert, set the arm for 180g.  Play a 160g record and then play it again with a 1mm paper makeshift mat.  See if you can hear a difference. .