"an amp operating withing it’s specifications that is producing 100 W RMS at a given frequency will reach peak power in the same amount of time as a 1 W RMS amp will."
You missed the point. I said ALL OTHER THINGS BEING EQUAL. So the same amp, be it 1 watt or 100 watts, is used with both speakers.
Obviously a 100 watt amp will produce 1 watt (0.278 times the voltage) quicker than it will produce the full 100 watts.
Even a perfect square wave has a "rise time" and obviously it passes through 1 watt on the way to 100 watts.