spatine, assuming the sensitivity you provided is at 8ohm, here is what I calculated based on your numbers (these are only best estimates):
at an average volume of 70 db assuming a peak of +15db, you will need 0.03W at 4ohm and 1.0W at 85db
at 81 db average you will need 0.5W at 4 ohm and 16W at 96db.
If you speakers dip down to 2ohms. then double the power.
As you can see, you really do not need a lot of power. For you to use around 250W of power at 4ohm, you will need to be listening at 108db!
In opinion, most people do not need high power amps. However, as it has pointed out, sometimes higher power amps have better circuits or designs which produces higher quality power at lower output.
Note that these calculations were obtained from a spreadsheet I developed which has some assumptions, approximations, and estimations. For example, I assume it takes double the power to increase the volume by 3db. I also assume that the volume decreases by 5db for every doubling of distance from 1m.