Every component in the signal chain has a rise time. The longer the rise time, the slower the signal transfer. It's true that there are faster components, amplifiers and interconnects, but all of them are way faster than speakers because they are only subject to bandwidth issues whereas speakers are also subject to physics--mass acceleration and deceleration.
Very fast components and cabling can improve the sense of speed somewhat. For example, if you have ultrawide bandwidth components (such as Spectral) with cabling to match, you *will* hear an improvement in speed and clarity. Better speaker seating and platforming helps too. But ultimately the speed is limited by the speakers. The acceleration of speaker cones is much slower than electronic signals. So far the only speaker that can produce a credible square wave (which is primarily dependent on rise time and controlled ringing/overshoot) is the Quad electrostatic.
Still, many new speakers use very light and responsive diaphragms with much more powerful magnets to improve speed. Also, an increasing number of tweeters are very fast, with beryllium deposit, plasma, or ribbon tweeters with frequency response out to 40-60KHz (indicative of excellent rise time).
The price of fast speakers has come down considerably. Consider the GoldenEar products that use a folded ribbon tweeter and a very fast cone midrange, priced from $800-3K/pair, and the Monitor Audio Silver series with CCAM (composite ceramic aluminum magnesium) drivers and the Gold series with ribbon tweeters. The newer speakers with lighter drivers and more powerful magnets will do more to impart speed than an upstream upgrade.