The design of The Cartridgeman‘s Insulator is precisely addressing that issue while avoiding your concern of cartridge displacement by choosing a polymer too stiff to be displaced by the cantilever, which displacement -as you point out yourself- is magnitudes smaller than the former. On my system (Zyx Universe, Dynavector DV 507MkII) the benefit is so obvious, that I can only recommend that people try it rather than defer to principle based pontification.If the cartridge transmits vibration to the arm, the arm hasn't done its job properly. But its a simple fact that no arm is perfect in this regard. So what really becomes the question is whether the arm tube should be damped to absorb such vibration or whether you want to decouple the cartridge from the arm in trying to do the same thing?
From an engineering perspective damping the arm tube seems the more elegant approach. But this is by no means universal so you may well perceive a difference (hopefully better). I alluded to this earlier.