How to make a true comparison?

Dear Pryso, aside from some small and fairly lightweight TTs I would always suggest moving the very same tonearm/cartridge combo from one TT to the other.
Any comparism which shall give a true result or at least an idea of the differneces should be made with as few parameters changed as possible - if possible the only variable factor being the component under review.

As for loudspeakers we can only compare speakers with similar demands as for power, damping factor, sensitivity and complexity of crossover-network.

Even comparing speakers against each other driven each by a suitable amplifier will not give true results if the two amplifiers aren't identical.
We would compare speaker/amplifier-set-ups, but not speaker vs speaker.

Our dilema - its very difficult to compare any singular opart of teh audio-chain for itself.
Best chances are with TTs and preamplifiers given fairly normal impedances in teh power amplifier(s).

The problem is that what the question is usually about is "how do two components compare IN MY SYSTEM". How they would rate in a system adjusted to make the most of their virtues would be in most cases irrelevant as the person asking the question is very unlikely to have such a system already or assemble one later. What he wants to know is how components will fit into his existing system. As each part of the chain introduces different variables each system will be to a greater or lesser degree unique so the most that can be achieved is generalizations. Turntable testers use to use identical arms and cartridges, transferring the same stylus between them in MM cartridges, even this did not produce an exact identical sound. Moving the arm between tables precludes quick AB comparisons. In the end making any meaningful comparisons is a tedious job requiring considerable skill.

Well personal preferences - which are fine if its only about "fitting in MY SYSTEM" - and interaction with other parts of the audio chain simply do not allow any comparism.
Unless you imply that all other parts are "perfect" you will never be able to tell or distill the whereabouts of certain changes in sound as they might well arise from certain interactions of components linked to tiny details in their technical specifications.

From the point of logic - if there is any such thing in high-end audio at all... - we need to eliminate all other variables to conduct any test or comparism with results worth mentioning and - maybe... - transferable.

If one is going for "which of the 2(3) components is BEST in MY SYSTEM" - that is a much easier task and is always a result of "fitting into the matrix".
And even here the result is only a status quo of the moment as the results may most likely change with the next "upgrade" in the system as the synergy-effects achieved before may have gone now.

Endless story - the results depends on the goal set and on the parameters of the periphery.
Stringent or freely?
Subjective or objective?
Viewed in the context of a certain system or seen individually?

Dear Pryso: +++++ " In the end making any meaningful comparisons is a tedious job requiring considerable skill. " ++++

I agree wth Stanwall but that " tedious job " is part of a serious comparison.

Your first part of the solution 1 is the way to go: everything the same but the TT.

If you can't do it then you are not doing a straight/right/precise comparison. Even if you can do it there will be a second variable factor that is the own each TT arm board that normally is dfferent and that has influence in the final result.
The best way could be to have same stand alone ( tower ) arm board. We are talking here of what you ask: " a true comparison " where the only change is the TT it self.

Regards and enjoy the music,
Raul.