A couple of comments. First, the position of the volume control on the preamp doesn’t tell you how much power is being used by the speakers. One is at the mercy of the recording volume, the output of the source/DAC/preamp, plus the input sensitivity of the amp and the efficiency of the speakers. I’ve seen systems where having the volume knob at 4 or 5 o’ clock gives a modest volume and other setups where 9 or 10 o’clock is overly loud and distorted.
Also, playback volume and power needed in watts have a logarithmic relationship. A 3 dB increase in volume requires double the power. Twice as loud requires ten times the power.
Hence, one of the critical issues revolves around an individual’s max preferred listening level. Headbangers with inefficient speakers in a large room need a LOT of power. Others, like me, rarely break the mid-80s in terms of dBs.
Like the OP, I found that I simply don’t need a lot of power. My class A Schiit Aegir (40 w/ch at 4 ohms) gives me way more volume that I ever need. Of course, having a separately powered subwoofer takes a load off that amp and the main speakers.
The key is to experiment with what it takes to make YOU happy.