Yes,
Sun rays ultimately change all that is in their path.
With speakers it will bleach the cabinets, dry the wood, and also, affect the drivers, (surrounds) if exposed too much.
Sort of like the white rat tests. "If a rat consumes 12.5 Million gallons of X it causes cancer". Hell if he consumes anything in THAT quantity it will kill him.
Moderation, combined with some UV filtering will help.