Best way of removing rust from silver or other

The tarnish on silver and copper is a form of oxidation (so is rust), but it forms a patina on the surface and does not actually destroy the metal in the same that rust will cause iron and low-carbon steel to decay and eventually crumble. Any good silver cleaner/polish will do a good job of cleaning your silver banana plugs, although you should be careful to remove all of the cleaning agent (Q-tips are helpful here).

There used to be a two-step contact cleaner from Cramolin. Don't know if it's still available, but it came in two small bottles, one red and one blue. It was wonderful for removing oxidation and laying down a thin layer of contact enhancement film that prevents further oxidation. You use it sparingly so the bottles last a long time. I'm still using the stuff I bought 10 years ago...

I use a Dremel rotary tool with a carbon steel wire wheel. DO NOT use a stainless steel wire wheel, it is too stiff and will leave scratch marks. I then follow up with SuperContact or similar contact cleaner. This process works great with AC plug prongs also.

Silver tarnish (oxide) doesn't affect the electrical properties of the metal at all. So, not to worry in that regard.

Ed