More watts or better power ?

I disagree with the generalization that more power is better. My Marantz 1060 at 30 wpc sounds better than most amps with 100, 200, 300 etc. It is all about the sound. It drives my outdoor speakers and they can obtain concert level sound with excellent fidelity.

As you are exploring the SVS option, take a look at the B&W PV1D. I just added a DB3D to my Magicos and the result is nothing short of spectacular.

The very best sounding amps I've heard are all low in output.  But, these amps are not appropriate for many speakers and in situations where high volume is a requirement and/or the room is extremely large.  So there is a minimum amount of power that is needed, and that minimum is situational.  

It is hard to say whether the 50 watts you have is sufficient.  All else being equal, more power will at least afford you more headroom for the brief peaks when playing very loudly.  That may or may not be a sufficient basis to upgrade to higher power.  But, often there is a sonic price to more power.  I once heard a comparison between a 50 watt Rowland amp and a very similar Rowland amp that was much more powerful.  I actually preferred the sound of the lower powered amp, and the speakers used in the comparison where supposedly a power/current hog.  I don't know why this was the case--it could be that using a larger number of output transistors adversely affects the sound, it could be that some transistor amps don't sound as good when they are just loafing at a small fraction of their capability--but this showed me that more is not necessarily better.

All of my current amps are low-powered tube units (6.5 watts/channel being the brawniest), but, I have appropriately efficient speakers and I prize good low-volume performance over ability to shake the room.

I agree with a-lot of the comments good input. I have to focus on two things. Great speakers for your room or small office. You will notice a remarkable difference with more power ie 200 wRMS plus.
It does sound that you have the upgrade itch. I have found that once you have the right amount of power everything else falls into place.
IMO your amp sounds great with more efficient speakers. 89db +

If you’ve a small room what is it do you feel your 50 watt Class A amp isn’t providing? I’m in a small 10 x 15’ room. I’m driving Martin Logan Summits with a 45 watt tube amp. I’ve grateful to find bliss in a small room. 

Hi Op,
I don't usually post here, but from reading through this thread your amp act as a voltage source, outputting 14 volts. Assuming that at dynamic peaks you are using 100 watts total, that would mean that your speaker cable has to handle 2 amps per channel. At a 30 foot run, in order to maintain voltage within 1% loss, you would need #8 awg cable for your speakers. Smaller cable will cause significant voltage drops at higher current loads limiting the dynamics of the amp. You've got great stuff, try either a shorter run, or beef up your wire gauge on your speaker cable.
Best Regards,
Mike