Unsound, I appreciate your last comment. It is where I was for many years, and to this day (and I expect for years to come) I still run into horn designs that are guilty of everything you mention in your first paragraph. When horns suck, they really suck. But like any technology, its all in the execution, and when you encounter good horns, they will lack most of the objections you have except size and maybe what it means to be ugly.
A special point with regards to
so sensitive that they amplify every minutia of noise and distortion
Really, every speaker must play every minutia of noise and distortion to be a good speaker, but the noise/distortion floor of electronics is definitely an issue with horns. When we first encountered customers with horns, I think we failed pretty badly on this point because our reference speakers at the time were 89 db. This forced us to really figure out how to get our noise and low level distortion floors down with the end result that we now build a much better product. That revealing nature is thus a boon to progress, but a bane if you have problems in the electronics.