Why are low impedance speakers harder to drive than high impedance speakers

To me it seems like if you need twice the amount of electrons to flow to a 4 ohm speaker than to an 8 ohm speaker, the amplifiers would need to work harder and in return wouldn't this cause more distortion?  Also, if it takes twice the amount of current to drive a 4 ohm vs an 8 ohm speaker, wouldn't this mean the 4 ohm speaker is less efficient.  However, you would think the 8 ohm speaker would be less efficient because it has twice the amount of resistance to the current.

I was told once a 4 ohm speaker requires twice the amount of current than an 8 ohm speaker.  Because a 4 ohm amplifier delivers twice the amount of current, would this in turn supply twice the amount of information to the speaker to create more detail in the music?

Since the OP thinks in mechanical terms, this analogy may best describe how speaker loads affect an amp.
Imagine the amp is a somewhat fragile flywheel that will fly apart at a certain RPM. If you apply a resistance (an 8 ohm speaker load in electrical terms) to the flywheel that is sufficient to prevent the flywheel from reaching critical speed it will not break. If you remove part of the resistance (think 4 ohm speaker) the flywheel will speed up. If you remove enough of the load, you eventually reach a point where the flywheel is spinning so fast it fails. Replace the concept of the flywheel speed with power output from the amp. The power an amp will produce is inversely proportional to the resistance to current flow. More resistance keeps the amp in check so to speak.

Hi all, 
This is my first post in this forum :)

According to Al's information, having the two extreme examples, 
I have the exact opposite question: 
For headphones, why do we need good, high, powered amp to 
move headphones with very high impedance (300-600 ohms). 
According to the ohm equasion, for the same power, as long as the amplifier "understand" that the output is not disconnected (as the "100,000 ohms" al's example), than the amp needs only a tiny current to "drive" the headphones.
What am I missing here? 

Thank you all!

For headphones, why do we need good, high, powered amp to
move headphones with very high impedance (300-600 ohms).

You don't. Many phones of this impedance are easily driven by a few milliwatts. Some 'phones require a bit of power and others don't. How much power they need is independent of their impedance.