My understanding is Df matters but there is a wide tolerance in values that will work in any particular application.
You have to listen and judge. At least with dynamic drivers, if the bass sounds mushy or soft, a higher damping factor may help but the actual value does not matter that much.