Important if it is off the rails in one direction or the other but, at normal values, somewhat less important, and....after a point, higher isn’t necessarily better, as others have pointed out.
It is simply a measure of the amplifier’s output impedance, which may be more important with speakers that have impedance variations with frequency, and/or large impedance dips. When you divide the amplifier’s output impedance into the speaker’s input impedance, that is your damping factor. Amp manufacturers often use "8" ohms for the speaker’s input impedance when they calculate their advertised damping factor. This may or may not be accurate, all or some of the time but they have no way of knowing what speakers will be used.
Damping factor is something to be aware of if you are changing out speakers or amps, but not usually a deal-breaker or deal maker.