What makes a speaker sound great at low volume?

I don't know if particular design attributes contribute to, or detract from, low level listening.  But, as a general observation, the best sounding systems at low volume tend to be high efficiency designs, particularly horn-based systems.  It may seem odd, but, those massive horn systems play better at low volume than at high volume (this might explain why so many Japanese audiophiles are attracted to horns even if they have neighbors that are very close by).

In part, high efficiency allows for the use of low-powered amps.  Low-powered amps, particularly tube amps, sound very lively when played at modest levels, while high-powered stuff tends to sound dead at low volume.

chayro: Love your first sentence ...

OP: I don't know if you're considering vintage speakers as well, to solve your low-level, listening issue, but I have a pair of Kef R107's, and they produce a wonderful sound-stage no matter where you are in the room. They are supplied with a small EQ adjuster that is electronically, part of the crossover. At low volumes I just boost the Contour adjustment dial. (This is the only time I touch the contour dial). This seems to add what's required to cover all the necessary frequencies, and the music sounds great.
Most people seem to like the 107s, but some just don't. They're a little different.
Anyway, just my two cents ...

IME, active, near field, studio monitors are probably the best bet.  Get the x-over ahead of the power amps and you’re most of the way home.  The best examples are quite expensive, but even many of the modestly priced units are very good at lower SPLs

     Interestingly, I noticed an unexpected improved low volume listening performance when I switched amp type from an Aragon class AB to a class D amp of about the same wattage.  The improvement was significant even though I was using an older pair of Magnepan 2.7QR large panel spkrs that are fairly inefficient (87db @ 1 W).  
     I'm familiar with 'loudness' controls and would equate the improvement to their affect; a perceived increase in the bass and treble to compensate for the proven reduction in perceived bass and treble when listening at low volumes.  
     What I found especially noticeable and beneficial with the class D amp performance was that the proper bass and treble presence maintained its proportional accuracy at all volume levels, from soft to very loud as if there was a 'loudness control' continuously engaged. 
      I cannot verify that all class D amps share this ability, however, I currently own 3 different brands of class D amps (ClassD Audio, Emerald Physics and D-Sonic) and they all share this ability or characteristic. 
     I would also agree with the several other responses on this thread that  stated moderately high efficiency speakers would be beneficial for improved low volume listening performance.

Tim
 

No crossover. Tube amp directly driving the woofer...

+1 .. my Decware/Omega pairing continually surprises me with the level of detail and dynamics when listening at low volumes levels.