I am prefacing this by saying that I manufacturer very high end ($10K) stand-mount speakers and it is my opinion and worth what you paid for it.
I spent years with a pair of Revel Performa3 M105s. I had put together a system where everything was about the same quality. Ayre Preamp, Rogue Amp and Marantz Reference DAC and Rega table. My feeling was that the speakers were the least expensive component, they were the next item that needed upgrading.
I went out and tried to find speakers under $5K that were better. I could find some that were different (Scansonic, DeVore, Acoustic Zen and Monitor Audio) all sounded different (and very good) but I wasn't convinced they were better. Some sounded much worse but they will remain un-named.
So I upped my budget and started looking in the $6K-$22K range. Differentiated sound was not exactly easy to find. I listened to some extremely expensive and underwhelming speakers. The ones that really stood out to me had something about them that was really unique:
Boenicke - look at these speakers for a second and it is obvious they are different. They are hard to place in a room, but are special when setup correctly.
Raidho - Tweeter is amazing. Sound is not good it is so amazing it becomes addictive. But, if your room is not open enough, the soundstage won't come together and they are underwhelming.
Wilson Duette 2 - Freakin' phenomenal speaker. Works well up against a wall and imaging is sick. But they are $22K dollars. Cast phenolic resin cabinets are what is special here and they are far better at damping than any wood/MDF.
Kaiser Chiara - To this day, the best sounding stand mount I have heard. Horrible to place. Rear firing woofer means they need 8' behind them. Cabinet is a rubberized composite of some sort. Not a hint of brightness from the mundorf AMT tweeter. Totally impractical, $22K but in the right space, my god they were amazing.
Magico S1 MK II, Wilson Sabrina - I also listened to these as small floorstanders that would take up same space and they were brilliant. Aluminum and cast phenolic resin cabinets.
Wilson has since introduced the Tune Tot. I have since heard speakers from Marten which are outstanding along with Vivid and Rockport speakers.
Basically, my POV is that if you have a wood or MDF box, you need to have an extraordinary tweeter like Raidho and mount it in a metal baffle or decouple it and mount it in metal like Vimberg. Alternatively, you could simply ignore all throughts on speaker design like Boenicke. Or you can use a non-traditional material to make your cabinets like aluminum, fiberglass, carbon fiber, cast phenolic resin, stone, cast iron, etc...
This is where you will see a big jump in sound quality due to the damping effect of the cabinets and where your law of diminishing returns on speakers breaks down. I make my premium products out of fiberglass and carbon fiber. They are expensive ($7500 -$10,000) but at least they are differentiated in terms of sound and you won't hear a small difference between a $3K speaker and one of mine.