What's striking to me about this thread is that most people who have converted to single driver systems say it's the "distortion" of multiple drivers and crossovers etc... that they are free from that provide them a clarity and magic that multiple drivers can't match. However,the pro-multi driver guys say that the "distortions" of a single driver trying to produce ALL frequencies is like nails on a chalkboard to them. OK, so where does the truth lay?
IMO/IME it has everything to do with how hard you push the speaker. If not hard, the single driver speakers will totally deliver! I've heard them sound amazing- especially in smaller (American) rooms (in Europe and other regions where smaller rooms are common, single driver speakers can excel!). But if you like to play at serious volume levels in a bigger room, and especially if you want the bottom octave or two, then you will find single-driver speakers to fall short.
I don't think it has anything to do with how we hear. It has everything to do with setup!