enough amplifier power


I am curious as to why so many people think that their amplifiers are powerful enough for their speakers. I use a Yamamoto A-08S--around 1.5 watts output. I use it with a Fostex F-106ESR. The combination is a little ragged at low volumes, but beautifully immediate. Distorts awfully at anything approaching a decent volume. I see people using 20-100 watt amplifiers with medium efficiency loudspeakers. I do not see how this can work any better. If you work out the math, most loudspeakers need 200-500 watts minimum. That is not even taking into account low impedance loudspeakers. Do people not know what distortion sounds like? Or, compression either, for that matter? Please enlighten me.
hedwigstheme
i used to read stereo review since the early 70's! this magazine said that you should buy a unit with the most watts you can afford! 
"...The golden rule is that your amplifier should be about twice as powerful as your speaker can handle..."

I’ve never heard such a rule. The reality is you need an amp that at a minimum gets the job done. But if you like to really jack up the volume, you better have a good amount of headroom. RMS power is meaningless because it's the distorted peaks that kill drivers.
OP here. I am not complaining about my amplifier. I am acknowledging the likelihood that it is not enough even for my efficient loudspeakers. I think a lot of us drive our loudspeakers to amplifier distortion on dynamic peaks. Agree or disagree is the question. Perhaps I could ask for Ralph Karsten’s opinion (@Atmasphere) since he has discussed this regarding SETs in the past. Perhaps he could take it further with amplifiers in general.
You know who is bashing some of the best sounding speakers on the planet to go with some of the cheapest sounding speakers while indicating that there is no difference in SQ between them. Far far from reality. Unless you are talking about the old large klipsch speakers, I can’t listen to any other klipsch for more than a few minutes. Same goes for tektons. There are so many excellent speakers out there that cost $1000 or more that are much better than these 2 brands, but with better speakers, you will need better equipment to get them to sound their best, wether the amp needed is a SET amp or a 1200 watt monoblock 
There was a thread a while back that went something like this,
"if you were starting over, what would you do differently"
If given a do-over, the one thing I would do differently would be to start with more efficient speakers that were an easier load to drive.  That would have opened my world up to many more good amplifier possibilities at more affordable prices.  Having said that, I did find speakers I like but they are a fairly tough load and need lots of power to really sing.  Several Class A amps couldn't quite cut it and I finally found what I needed with a pair of 650 wpc monoblocks.  Having that big power does make a difference when you feel like cranking it up for that closer to real experience, but also at moderate listening levels, IMO.