How long does it take a decent quality speaker to "wear out"?

My 35 year old Spendors and 56 year old Quad ESLs still sound excellent. 

My 42 year old Infinity RS-1.5's are still going strong. The woofer and midrange surrounds have been replaced. Otherwise, trouble-free and sounding fantastic...

There is a very wide range from about 10 years to infinity. With regular dynamic speakers like yours 30 years would be about the norm. Certain types of loudspeakers like modern ESLs can be virtually immortal. 

It depends on the materials used....some can go on for lifetimes. Foam surrounds rot and will need to be replaced usually within 20-25 years. Some caps will also need to be replaced at 20-30 years, others will go much longer. Ferro fluid will dry out in time and need to be replaced in 20-30 years. With some maintenance, many can live on as good as new.

My personal speakers are pushing 33 years old with components that are over 35 years old. The butyl surrounds and kevlar cones are in great shape, as are all the crossover parts. Solder joints all look good. If anything, the well flexed spiders and surrounds work better than ever. Even the paint and cloth grills still look great.  The only thing I’ve replaced was ferro fluid in the tweeters.

Durability and lifetime depends on specific design.  I have heard speakers with 70 year old drivers that sound terrific.  Paper woofers with pleated fabric surrounds can be quite durable compared to even modern woofers with rubber surrounds.  My midrange compression drivers are about 80 years old. Even speakers with surrounds that go bad can be long-lived if you accept that surrounds have to be replaced every twenty years or so.