What does the term "Speed" mean in a speaker?

Impression being the operative word. Probably.
In reality, and measurably, no.

Fast with a speaker for me refers to low distortion resulting from minimal time lag due to inertia between signal and movement of the transducers as would be expected with a good combo of stronger magnets, precision machining and assembly of parts, and low mass driver and voice coil.

Result is the transducer/driver moves faster and more accurately than usual in response to the electric signal provided.

Efficiency is different.  That's the SPL produced in response to a standard electric signal.     Louder does not mean faster or less distortion in the time domain necessarily.

simmonmoon is correct. The correct term is transient response. Speakers with faster transient response tend to be more detailed and dynamic.
There are a bunch of rather silly terms out there in regard to audio equipment sound quality. Terms like pace and timing are frequently misapplied to speakers. The pace and timing of music is set by the musicians not the speakers. More poetic waxing.

+1  transient response is a proper technical term.

https://en.wikipedia.org/wiki/Transient_response

"fast" is slang.

Yep, transient response. I would say decay does not fall under the umbrella of speed or transient response but others may feel differently.