Let me clarify a little.
Each driver has a different distance to it’s acoustic center. Tweeters are shallow, woofers are deep, and we must take this into account in designing the crossover.
Here’s the problem. From the front the tweeter is say 2" closer to your ear than the woofer but at 90 degrees to the side they are the same distance. This means that the phase relationships you take into account in the crossover only hold perfectly true in front of the speaker.
If on the other hand we step the drivers back so that the acoustic centers (usually near the magnets) are exactly aligned, the phase relationship directly in front and at the sides is the same.
Great! this simplifies crossover design. However, if you look at an impulse response in either case, they will still not be perfect. That is more involved. :-)
Another approach here is also to use higher order filters, which D'Appolito recommends for MTM designs. Steeper filters (also like Joseph Audio's older infinite slope speakers) minimize the phase matching problem off axis in all directions.