From what I’ve read, and experiences from my attempts at building speakers, it does seem to be important that the overall power output be smooth, with a downward tilt of some sort toward the treble. It really is difficult to get a speaker to simultaneously have a flat, smooth on-axis response while also having a smooth but downward tilted response off axis at all angles. There’s a ton of stuff that goes on between the drivers and cabinet baffle, port, internal enclosure resonances, etc. interacting with each other that make getting the off axis response smooth at all angles a can of worms. If you can’t solve all of that, then some compromises here and there might not be all that audible, while allowing the speaker to perform better in terms of lower distortion, extended response, more headroom, etc. The ideal dispersion pattern for a speaker is not something that has been settled, and may vary depending on tastes, the characteristics of the room being played in, and perhaps the type of music being played. What does seem to hold true though is that whatever the dispersion pattern, narrow or wide, or somewhere in between, or perhaps various combinations of wide up to a certain frequency before narrowing at various rates, the measured response from any angle off the speaker should ideally be smooth across the frequency band, with no stand out peaks or dips. If there are some peaks and dips, ideally they should be random at different angles, and not the same at all angles. However, if they are the same at all angles and you have access to a good EQ, you may be able to improve that speaker quite a bit with EQ.
I think there’s a limit to how smooth and flat the response needs to be, at least for me. I’ve recently compared speakers that measured better in every respect to what I’m using now. I even think they sounded better as a result, but they weren’t really solving anything that my ears couldn’t easily adjust to. My ear is forgiving of certain flaws, and I may even find some subtle colorations charming, so I was not persuaded to move up. My imperfect speakers are close enough, and I think I prefer their slightly wider dispersion in the upper frequencies, even if it isn’t as smooth and even.
Of the various speakers I’ve built, I’ve found that I really liked a minimum baffle design that used a 3/4" tweeter and a 4" woofer. I never tested the response on these, just set the electronic crossover by ear and found them captivating. My girlfriend did too. I’ve also built narrow dispersion horns that could hold a fairly tight pattern down to 600 Hz. I liked those a lot too, played in the same room as the high dispersion speakers. What I think I don’t like are speakers that are narrow up top but don’t hold pattern low enough. If you can’t hold it down to 600 Hz, then keep it wide as high as possible.