There is no substitute for cubic inch or in this case surface area

We had a 32" Clarion sub in our possession for quite a while.  In a 6 cu ft sealed enclosure it would rattle the rafters in our (and adjacent) building -- while moving about 1/4". 

Thoughts on the OPs observations:

What might be happening here an aggregate of amplitude, detail, and dispersion. While smaller drivers are very good at projecting an image from wall to wall, they may be a couple of dbs short of a glorious presentation.  And, have to work pretty hard when dynamic passages demand that the drivers to move large distances. When you have large bass drivers of dipping well into the subsonic region and producing massive amounts of bass, the speaker designer can "open up" the other drivers (or choose higher efficiency drivers) and let them play, thus increasing the dynamic range of the entire musical bandwidth, and lower overall distortion.

At lower levels, the system is just humming along with low distortion and plenty of dynamic headroom on reserve.  This lower distortion also keeps spacial cues intact, thus providing a coherent soundstage, while maintaining detail and focus.  In this way, the speaker with more "cubic inches" may, in fact, provide a more realistic presentation than a smaller example.

Bigger can be, and in my case is better.  Just scored a pair of Pure Audio Project Trio 15 coaxials at AXPONA25.  Two 15 in woofers per channel.

I haven't been this immersed in the music since I got my Thiel CS3.5's. I also have a pair of Ohm Walsh 4.5000's.  Each of these speakers do some things better than the other and I'd often switch between them.

But the Pure Audio Project Trio 15 coaxials do it all better.

There is no replacement for displacement. 

My listening room is 20'x15' with wooden floors (with an 8x12 carpet) and windows. What I don't want are woofers that are so powerful they rattle my windows and shake my floors, adding bass distortions.

I have a pair of Sonus Faber Olympica Nova 5s with three 7" woofers that go down to 32 Hz. They are perfect for my room. I can feel the vibration on really low sounds and various percussion like tympani, sticks, and snare drums sound accurate. I can also hear placement and depth.

Prior to these I owned a pair of Golden Ear 2+ speakers. They had two 5" by 9" subwoofers and two 8"by 12" planar bass radiators. The bass was not nearly as accurate as my Sonus Fabers. I could turn the bass up and down, but if I turned it up too high my windows rattled and floor shaked. So, accurate with sound waves that are felt by my body is about as much bass as I want.