It's up to the designer to balance the off-axis response, with the crossover, to yield the best tonal balance.
As Graveson has noted, sometimes we design speakers for listening at the tweeter axis and find we prefer listening one driver down. Some speakers are deliberately designed with mid-woofer over tweeter, and designed to be llistened on mid-woofer axis.
This is a good experiment with most speakers, try listening below the tweeter. :)