Most (not all) SS amplifier has relay at the output for delay turn-on and protection. If both tube amp and SS amp are connected to the speaker and only the tube amp is on, it will do no harm to both amp because the SS amp output stage is disconnected by its built-in relay.
When the SS amp is on and the tube amp is off, then the SS amp will see the speaker as a load and also the tube amp output transformer secondary winding parallel to it. That will be bad for both amplifiers, there’s possibility fries the tube amp OPT and damage the SS amp or put it into protection mode.
Even though if the tube amp OPT secondary winding strong enough to stand the current from the SS amp, the speaker will definitely sounded weak with no bass because the inductance of the OPT secondary winding drained all bass energy through it.