Speaker manufacturers or box manufacturers with off the shelf drivers?

The design is as important as it's execution. And too many do not ever address even the easy details. Let alone the hard one's.

I Think it comes down to being a smart consumer an educated audio enthusiast and training your ear/brain. Speaker design is an art and a science and its up to you to decide if a speaker represents a good value for your budget and tastes.

Also don’t forget that driver designers are experts at what they do and make them often at an economy of scale that would be much more if they were bespoke for each speaker company.

To use a car analogy, a lotus Elise
I once rented had a Toyota engine and transmission. It was a fantastic exhilarating ride and if I didn’t live in a NYC I’d probably own one. Did the Commuter car components (though tweaked) take anything away from the car? No, in fact it made it affordable and reliable.

This is fascinating.  I am a small speaker manufacturer and drivers to me are one of the thing that stressed me less.  There is a lot of work that goes into choosing the right driver in terms of testing with your cabinets and materials but there are lots of bigger manufacturers that use stock drivers that make AMAZING speakers.  
To name a few, Wilson (small mods in some cases) Avalon, Marten, Joseph Audio  all use stock drivers.  These guys make some of the best products in the world and they use stock drivers from Scanspeak, Eton and Accuton, Accuton and SEAS from the companies above. 
I have been told that Revel has since moved production internally but the drivers in The Performa3 line were SB Acoustic when it launched.  Doubt this?  I still have a pair from years ago with SB drivers in them.  The tweeter had a waveguide designed by Revel.  That was what made it their own.  
There are crappy speakers out there and you can spot them and avoid them.  Check out this series of articles from audioholics:

https://www.audioholics.com/loudspeaker-design/high-fidelity-loudspeakers 

See links at the bottom of the page for other relevant topics like this one on the explicit elements of a driver:

https://www.audioholics.com/loudspeaker-design/loudspeaker-drivers

in the series they touch on all aspects of speaker design other than utilization of exotic cabinet materials (aluminum, fiberglass, carbon fiber, stone, etc...) but those materials are clearly superior to MDF based on guiding principles outlined in this series.  
I will admit that i designed my products so that if Audioholics took them apart, they would be pleased with what they found.  
Good luck to you in your shopping.  

Most of the speakers I have owned and liked, with one exception, make their own, especially Magnepan.  The only speakers I have owned with off the shelf drivers were Dahlquist DQ-10,  

I am sure if I were to start a "speaker company" tomorrow, it would be easiest to build a box and use some ready made dynamic drivers. 

Lets bad rap the world since were home an board.