How long does it take a decent quality speaker to "wear out"?

Film capacitors, inductors and resistors could last practically forever if not abused. Electrolytics after 20 years.

Assuming you never play it hard what will wear out is the surround and spider, that is, the suspension components. The modern synthetics are a lot better than the textile/cloth based surrounds but still I’d expect they’d be the first to do. Next question is, are they constantly exposed to direct sunlight?

Another risk factor is the cone material itself.  Paper drivers can absorb moisture and deteriorate or dry out and crack.  Magnesium (untreated) can also be subject to corrosion. 

@erik_squires

I used to run those speakers hard and loud. Quite often at ear bleed. There were times that I was surprised that they did not explode even though they never clipped. We are both older and I may be somewhat wiser and they are in a small listening room, so by my standards, they live a pretty soft life now.

They were never exposed to much sunlight at all, and now they live in the dark. No sunlight.

. . . @erik_squires  , if I remember correctly, B&W claims that the midrange drivers are made of Kevlar & the tweeter of titanium.

The only problems I've ever had with speakers were tweeters blowing out. It actually happened to me a couple/three times. With my Celestion SL600's and my KLH 17's.  I never played my stereo very loudly so I have no trouble blaming the componentry. And then there were several times when ultra-weighty "audiophile" speaker cable pulled my petite mini-monitors off their stands onto the floor. More than once it resulted in speaker cabinet dents & dings. For a while I'd rope speakers to their stands with twine. Now I use Quake Hold Museum Putty.