Output / Input Voltage Question

.5 volts is an unusually low input sensitivity for an amp. You can put an attenuator between the amp and preamp to fix the problem. This Schiit SYS will work!
https://www.schiit.com/products/sys

Can't go past 40% volume without being way too loud.

Why didn't you say so? All this voltage stuff is irrelevant. Your problem is you think that where the volume knob is pointed means anything at all. Relax. Where the volume knob is pointed means nothing at all. Or if you want you can tell people your amp is so incredibly powerful look how loud it is and the volume is barely cracked open. But seriously, where the knob is pointed is utterly irrelevant.   

This is in stark contrast to the numbers on the dial, which are incredibly important.  https://www.youtube.com/watch?v=4xgx4k83zzc  

That's a joke, son. (I have to say that because audiophiles have no sense of humor.)

@millercarbon LOL!!  I read all of these posts about voltage and impedance and I can't wrap my head around it.  Add speaker sensitivity into the mix and I'm completely lost. 

I'm in the same happy boat where noon is off and 5 o'clock is LOUD. Not a big deal except the less carbon track the signal has to pass through, the better. That's why a switched resistor type is better.