Small-ish Speakers That Come Alive At Low Volume

You should audition the Merlin TSM's. Fabulous sound, all that you are seeking, and compact. I have had many speakers of all shapes and sizes and Merlin's are a keeper for me.
Happy Listening!

Fritz Carbon 7's, simply amazing, only $1750 but better than any monitors I've heard under $4k, and i've heard a lot. I would seriously take a look into it and contact Fritz for advice. No affiliation here, just want to spread the word on a gem of a monitor.

www.fritzspeakers.com

What is the room like? Are there a lot of ambient noises that will interfere with listenig at low levels?

Since a small-ish form factor is important, that rules out electrostats. I would stay away from multi-driver speakers and maybe focus on high efficiency speakers like Spendor. Your amp would be overkill, though.

A little lower on the sensitivity scale, consider a pair of JM Reynaud Twin MK IIIs...if you can find them: A most excellent speaker that does well at lower volumes.

I'll second the Joseph Audio rm7xl