Which Component Has the Greatest Affect on Low-Level Listening


I'm looking to get out of HT 5.1 and go strictly to 2 ch.  I generally have music playing all day, often just sitting done to listen to certain tracks.  

I've read that "some" speakers don't sound "good" until high listening level, and also an integrated such the Luxman 505 don't sound good at low levels (is that the reason they include loudness?).

Which component, the amp or speaker, has the greatest impact on low level listening quality?
bgm1911
Higher efficiency speakers are far better at low volume levels than power hungry higher spl speakers. You can overcome some of it but there are physics behind it as well that you can not overcome as well.
I would look for a good class A amp that will not change over to A/B until 5 watts. Low level class A sounds really good.
I’ve had very good results at low volume, with 97db horns and about 4-5 watts (300B).
mapman Interesting answer about sub, I think trying a REL 9Ti with my stereo gear.  Thank you
Start with a very low noise room.  A rule of thumb is that your ambient noise level should be at least half of your average music level.  It's hard to get below 35dB ambient.