How many watts??

A few good watts with an SET amp will drive efficient speakers beautifully. And there are plenty of efficient speakers to be had. Huge power is only necessary with inefficient speakers and the higher the power, the more circuitry, and the harder to produce lifelike sound. 

I can drive an inefficient speaker with 44w/channel and it sounds great too at the 12 o’clock position… (The speakers are 84 or maybe 86 dB/W/1-meter.)
If the OP is playing at 1/4 of the way up, it is not likely that they are playing it at more than 1W, and at that point… then what is the fascination with the speaker’s sensitivity and efficiency?

 The "OP" probably never gets past that 1st watt....
Almost all here fail to recognize just how much actual energy a transducer can output at (1) full watt. OR just how often all that they are  listening to normally?  All is supplied by that 1st incredible watt. "For many".
Even with speakers rated (85db) sensitivity!

millercarbon:

The great Robert Harley famously said, "If the first watt isn't any good why would you want 200 more of them?"    

This is wisdom.

Well I guess you did say "almost" all.

If I have a 80 watt stereo amp and I normally listen at quarter volume and never play at levels higher than this. Do I really need 80 watts could I use a 15 watt amp at half volume?

No, it depends on the input levels and your amp's gain. If your input level is very high, you could exceed your amplifiers output even when your volume knob is lower than half volume. It the input is very low, you could possibly not even come near your amplier's total output even with the volume turned all the way up.

There is no "golden ratio" between amps.

... if I have speakers that my mfg states needs min 50 watts to really make them sound good with my current amp rated at 80 watts but played at qtr volume does that mean I am not really getting the best from these speakers.

No, the manufacturers are engaging in a CYA exercise. The "best from your speakers" is what sounds the best to you. Don't sweat it, sit back and enjoy your sound. You are over thinking things.