How do you stop house guest from damaging your speakers?

Definitely tell any visitors not to touch any of your equipment. I guess I would preface this with mentioning, you have a really unusual and delicate equipment.

But I am also on the side of @bigtwin. In twenty years I think I have had seven or eight people in my audio room… most other audiophiles. My system is for me. Most people don’t know it is one of my pursuits.
 

For a long time I would mention I was an audiophile and I would hear. “Oh, I (or my husband is)… I (he) have a Bose system…” the shock and disgust I would experience was difficult to contain. I would try and explain that was not an audiophile system without being condescending… but it would leak. I finally just stopped telling any one I had a high end system.

I’m pretty direct, especially around cleaning people. "STOP!" is a word that comes out of my mouth pretty easily. :)

Education however works best. Before they get the chance to I point out that there are expensive and very sensitive parts not to get near. Certainly not to put a vacuum cleaner near, or their fingers. :)

It does drive me crazy that I've had so many people immediately want to touch what they see as unique.  The center plug on ring radiators for instance, but it's so natural that I have to just prepare for it when letting people near.

Heck, I keep my speaker grills attached even when I just play the stereo for my lonesome self. I also got to say that the few guests I’ve had in the stereo room were either too polite or too weirded out to put their fingers on the one component they could actually do damage to -- the turntable.

I have a keep out sign I had made that hangs on the doorknob, also a protected by Colt sign hanging on the other doorknob. They are hand carved by a Mennonite in Waco Texas. 

Wrap your gear with an electric fence. Just be sure to put plenty of noise suppression devices on your audio gear. A "please don't touch" sign might work as well, with fewer injuries to your guests and your equipment.