I rarely listen above 75db so a speaker's ability to sound its best at low volume is critical for me
The world is your oyster.
75 db is moderate (only about twice as loud as conversation). I think your current Ushers are way overkill if you listen at these levels. Your plan to downsize makes eminent sense.
At these levels, unless you are using a "loudness button" then you may want some bass boost...remember that 20 Hz is inaudible at anything below 75 db.
Good Luck!