Will sun exposure damage my speakers?

Yes,
Sun rays ultimately change all that is in their path.
With speakers it will bleach the cabinets, dry the wood, and also, affect the drivers, (surrounds) if exposed too much.
Sort of like the white rat tests. "If a rat consumes 12.5 Million gallons of X it causes cancer". Hell if he consumes anything in THAT quantity it will kill him.
Moderation, combined with some UV filtering will help.

Definitely look what it can do to human skin....

Sun is no good for speakers stay away.

Get yourself some liquid latex.  Foam surrounds that are failing absorb it like a sponge.  A few coats works wonders and adds years to the drivers. 

Yes keep them out of the Sun. As an engineer of 20 years in the polymer field you should keep your speakers out of direct sunlight.