If only a little is coming off or fraying, you could first try an iron (set to low) and run it over the speaker with a few sheets of paper between the iron and the speaker. This may melt the glue an cause it to stick. This may be your best bet since the glue may be weakening further back. You would still have to use clamps to apply even pressure along the repair. F-clamps with plastic protectors work best. If the repair area is longer than the clamp footprint, use cauls made out of 1x lumber with the clamps. If this doesn't work, then yellow carpenters glue and clamps will definetly do it.