How many watts??

Nor a Klipschorn with a 200 wpc amp!

If I have a 80 watt stereo amp and I normally listen at quarter volume and never play at levels higher than this. Do I really need 80 watts could I use a 15 watt amp at half volume?

My next question if I have speakers that my mfg states needs min 50 watts to really make them sound good with my current amp rated at 80 watts but played at qtr volume does that mean I am not really getting the best from these speakers. If I play at half volume this is too loud, do I need a bigger room. Sorry if I did not explain clearly enough.

An oscilloscope might tell you what the maximum voltage is you are presenting to the speakers. Once you know that, then you could make an objective decision.

Or just hook up your amp and subjectively determine if it sound good to you.

In general more power is better. Watts are relatively meaningless if you desire good quality sound. Lots of current is a better indicator. If an amp, say, doubles the wattage with decreasing impedance it is a high current amplifier. But basically the higher quality the amplifier the better they sound. There are different designs that sound somewhat different. 

"...Some manufacturers even build power hungry speakers just to sell their big watt amps…"

You got that backwards. Loudspeaker manufacturers like Magnepan and Infinity made loudspeakers that just ate up amplifiers. Then the race was on to see who could build big arc welding amps. Some of the first players were, Ampzilla, BGW, Bose1801, and others. 

In reviewing your question, I believe you are focusing too much on the OEMs guidance for the speakers.

Speakers can be damaged by 'clipping' which can be done by an amplifier that is below the recommended wattage rating.

I have a VU meters on my amp and when playing music in the background when guest are over for dinner the amps rarely deliver more then 3-5 watts.  I prefer to listen to music loud when alone and staying around 100 or so watts requires my volume knob to be in the 60-70% of full volume range depending on the source and the recording.

Therefore, are you enjoying the sound?  When you turn the volume up or down slightly are you noticing characteristics change in tone, timber and tempo?