Which Component Has the Greatest Affect on Low-Level Listening

No component, except one that includes loudness or tone controls.

Speaker response changes a bit with volume, but any competent speaker will not change even remotely enough to compensate for how much the response of your ear changes with volume.  If your amplifier has high distortion at low volume, you could loose some detail at low volume. Could happen with a pre, but unlikely.

It's your ears/brain that are changing and they are changing by a large amount. Only way to compensate is to change the response of your system.   This is one of the problems in audio. Different listening levels all present a much different balance of frequencies.

https://www.google.com/url?sa=i&url=https%3A%2F%2Fwww.kmuw.org%2Fpost%2Floudness-and-fletcher-mu...

Hi,

No doubt that more dynamic is available, more the listening to low level is confortable...
Try to compare mp3 vs cd or hi res..
Physiologic correction with loudness ( fletcher)remains artificial but efficient compromise  with hifi standart components..

As far as the amp reproduce the "matter" I would say that electronic is involved in the result..

Raymond

There is nothing artificial about compensating for loudness contours.  Not compensating could be argued to be the most unnatural wrt the original.

Which makes me ponder if maybe the Loudness button on my old NAD 3140 back in the early '80s didn't have a buffer-amp in the circuit?

There's a truth in what several are describing here, and I really appreciate Raymon's comments about convincing dynamics at lower volumes making the experience enjoyable.

I've heard Preamps and similar buffer amps (taking the upstream VC out of the equation) make such changes in a system. Fullness of dynamics, tone & spacial characteristics at all volumes. 

The preamplifier, doubtless.  Why?  Because unless it is forty or more years old, it lacks a "Loudness" control, which was specifically designed to solve this problem  It emulated the Fletcher-Munson curve in reverse according to the level set.  So you got all the frequencies regardless of level, if you wanted that.