Is more amp power always better...?

The SET folks say that the first watt is the most important. Some of them get by with 5 and I’ve heard even less. 
 

In other words, quality over quantity but you need enough to drive your inefficient speakers. 

Definitely not always.  We also can't really say this is true about having more damping factor. 

The amp/speaker match up is important as is the quality of the first few watts of power.   There are definitely speakers with troublesome impedance curves that like big amps with low output impedance, but also large speakers with big woofers that can benefit from high output impedance.  See Nelson Pass's writing for the latter.

So the ASR Emitter 1 might be a good option. I own the ASR Emitter 2 Exclusive and they sound great with a pair of Quad 2905’s. Quad 2912’s would be the same. I have to share with you however my experience when I matched the ASR with a pair of Monitor Audio GS 10 bookshelf speakers. The amp was just too much for the bookshelves. It sounded like a Tasmanian Devil was trying to break out of the cabinets. When I bought the Monitor Audio’s I was told that it was impossible to blow them up but I blew both a woofer and tweeter. I then paired the Monitor Audio’s with a Rega amplifier and they were fine.

The numerical value given for damping factor is 8 divided by the output impedance of the amp—the lower the impedance, the higher the damping factor.  A typical SET amp can have an output impedance of 2 ohms or more, for a damping factor of 4 or less.  This relatively high output impedance interacts with the impedance of the speaker, which varies with frequency, and thereby affects the frequency response of the speaker (the lower the speaker impedance, the greater the change in frequency response.  Thus, SET and other tube amps with high output impedances are best used with speakers with a rated impedance of 8 ohms or more, and with an actual impedance curve that is relatively flat.  Any damping factor above 8 is probably high enough to be irrelevant.

My thinking, a speaker having a true 89dB sensitivity to be used in a 10'x11' room can easily be by served by an amplifier providing 50wpc at 8ohms and 100wpc at 4ohms. 

My advice, search for an amplifier that mates best towards the presentation you desire rather than the power it delivers. Low watt SET amps an exception.