Aren't most speaker posts gold plated? If so, aren't we all running into a dissimilar metal mating regardless of what we do?
I know that many prefer bare wire and that may sound good for a while, but it is very difficult to keep clean. I thought that one of the primary purposes behind spades (etc) is to provide an easier surface to maintain since the spade connection to the wire should be airtight. Given that, the optimum material may depend on your habits. Gold is easiest to maintain. Both silver and copper require constant maintenance -- if you don't think you will clean the terminals monthly or quarterly, then silver and copper will become increasingly less articulate over time.
To aummarize, all conductors sound a bit different. If you keep the terminals clean, go with copper, gold or silver depending on your taste. If you don't keep them clean, go with gold. BTW, the impact of gold may be less than supposed since you are matching to gold anyway via the terminals.