Why hasn’t B&W ever entered the ultra hi-end for speakers?

It's most likely a business reason.  B&W has a set price area for their product.  They also have a large amount of capitol in custom manufacturing equipment.  Couple that with an already set distribution network.  As long as they can keep producing and selling, there's no real reason to jump into the "insanely expensive" category.  And yes, point of diminishing returns is definitely a factor.  If they marketed a new speaker at the $80k-150k level, the amount of sales would be significantly less.

I actually think they do amazing things with their current speaker design at the price point (diamond tweeter, curved back custom manufactured speaker cabinet from multi-layers of thin wood, etc.).

They already put what is arguably the world's best tweeter in their 800 series. Their top-level midrange drivers are hard to beat as well. 

They had.  Nautilus was the best speaker ever made.

The powered version of the 50 is even reasonably time and phase correct..... looks like some of the Junior Engineers have been listening..... there is hope

B&W speakers are an ok speaker, not anywhere near the best, way overpriced for what you get. They have always been too analytical for most and that’s why you see them teamed up with amps from Classe and McIntosh, your warmer sounding amps. They were noted for their midrange drivers not their tweeters.