I've been saying it all along. You have test it in time domain. The frequency domain in steady state response will not tell you much.
No you do not have to test in the time domain, for almost all tests. If the transient response is the same every time you apply that transient, which is true in most cases, then frequency domain analysis is totally accurate. Even where you may have periodic effects which you may think will impact the transient, such as AC line noise, frequency domain is sufficient. Even for speakers where we have power compression, which could be considered a transient effect, the speed of power compression, a thermal effect, is below our hearing range, that frequency analysis is good enough. For speakers, the transient response is almost always plotted in tests.