Yes, the woofer does try to play lower frequencies, unless there is a crossover cutting off those frequencies lower than a certain point, say 30Hz. Usually the bass driver is left to attempt to play those lower frequencies, but is less precise as the frequency drops.
If you look at the graph of a woofer's response you will see at the left side it will fall off, as though a steep hill to the left. This is the visual of the speaker's worsening in performance of the lower frequencies. It may perform at +/-3dB at 40Hz, but at 30Hz may by down to -6dB or so. The more veritcal the line as the curve turns down the worse the performance in terms of capacity to play the lower frequencies.
Do not expect a speaker with specs like 40Hz +/- 3dB to do well at 25 or 20Hz. It will not. You will not get the lowest, deepest bass in abundance from such a speaker, no matter what anyone tells you in selling it. If you want lower bass, bass you can feel, etc. you simply have to find a speaker with the proper drivers/specs. At this time, while I can still move around big speakers, I choose not to use a speaker with specs like 40Hz for my reference. There are many speakers with just as good sound quality and far better lower end, say 16-20Hz performance, though you will usually pay a lot more for it. It is relatively easy to find competent speakers with bass specs such as 25-28Hz +/-3dB, and that would give you a lot of low end. Again, if someone is showing you a smaller tower speaker, or a bookshelf speaker with smaller drivers and a rating of 40-45Hz without any tight specification like "+/- 3dB" avoid it if you want powerful bass; it will never give it to you. It will perform more poorly if the number is higher, such as +/- 5dB, or even worse +/- 10dB. A very tight measurement is +/-2dB.
Having said all that, this doesn't tell you what the tonal character of the speaker is, which is why you do not typically want to buy a speaker unheard. The type of cabinet used, and whether it is ported or a sealed enclosure design of a box speaker, and the crossover network also vastly influence the sound the bass makes. So, you will want to hear a speaker you are considering purchasing if possible.
If you go the small speaker route with less bass you can always do a sub.
But, yes, any bass driver attempts to go lower along with the frequencies of the media unless cut off by a crossover. It would also attempt to go higher as well, but the crossovers limit it. Regarding a driver reaching "infinity", it will distort badly as it is pushed harder and attempts to play frequencies increasingly outside of it's optimum range of operation.