Which Component Has the Greatest Affect on Low-Level Listening

Speakers make the biggest difference, followed by amplifiers.  Horn systems are particularly dynamic and engaging at low volume.  If you look at Japanese audio magazines, it is common to find serious audiophiles putting gigantic horn systems in their small homes and apartments.  Given the close living conditions and cultural emphasis on being polite and considerate, systems cannot be played very loudly, hence the desirability of horn systems.

I find amplifiers also play a big role.  Tube amplifiers, particularly low-powered tube amps, deliver lively and engaging sound at lower volume levels.  I find output transformerless tube amps to be particularly dynamic sounding.

Bass boost or loudness control is the right idea but powered sub is better. It provides extra muscle to provide the power hungry bass without taking it away from the rest. An amp driven less hard typically sounds better with lower distortion. Power demand increases exponentially as frequency lowers. It takes a lot of power for flat response to 20hz even at lower volume.  Especially in a larger room. 

To me super efficient speakers driven passively by say a Schiit Aegir 20w Class-A. are the way for low level listening, "IF" you can handle their colorations/distortions, I can’t, "especially" horns.

Cheers George

Music tends to sound thin at low volumes, that's why the Fletcher-Munson Loudness curve was developed.  Most higher-end stuff doesn't have a loudness control, nor does it have tone controls.  For me personally, the addition of the Schiit Loki allowed me to tailor the sound both for different headphones as well as for speakers at low volumes.  BTW, the suggestion for power conditioning and power cords is pure nonsense.

You answered your own question while asking it.

It is amplifier with a loudness control.

It is admirable that amplifiers strive to provide a linear frequency response.  But that is completely the wrong pursuit.  Engineers knew this about 50 years ago when they were designing amplifiers.  Since humans listen to them, and human ear has a NON linear way of hearing frequencies, all amplifiers made for humans should all include a loudness control.

THAT is the most important component.  It can be adjusted to overcome the shortcomings of a turntable, speaker and what not.  You cannot say the same for any other component.