So if you have an in-room bass peak in the subwoofer region, you will also have modal ringing at that frequency - the energy will there will take longer to decay into inaudibility. (It is the frequency response peak that we actually hear; the ear as very poor time-domain resolution at low frequencies but arguably has heightened resolution in the loudness domain.) The good news is that when you fix one (by EQ or bass trapping or some other technique), you simultaneously have fixed the other.
Therefore, in my opinion, the phase adjustments on a subwoofer amp (and/or timing adjustments within a bass management system) are tools to be used with the focus on getting the frequency response smooth. Smooth bass = "fast" bass. In other words, don’t sacrifice frequency response in pursuit of what you think would be better phase response. In the bass region when the frequency response is right, the time-domain response (which includes the phase response) is also right.
Duke