"Fast" or "speed" should refer to rise times and settling times of impulses (as it regards individual components and the system as a whole, including the room). Personally, I suspect that for transient attacks, it is really more a lack of dynamic compression than rise-times per se that gives an impression of speed. IMO it is actually the decay aspect that most distinguishes "fast" from "slow" sound, where overhang or resonances that obscure clean impulse decay leads to an impression of "slowness" (maybe in conjunction with compressed attacks and/or an overly reverberent or strongly modal listening room).