It is mostly about the implementation. Have you heard any of the Charney Audio speakers using such drivers? These are surprisingly complete, smooth, and well balanced systems. I like systems using such drivers as wide range drivers in multi-way systems, such as the Cube Audio Nenuphar Basis (powered woofer plus a fullrange driver).
However, I have also heard some pretty rough, peaky sounding systems using these types of drivers; it is a matter of finding a good implementation of this approach to making speaker systems.