It appears it is the speaker, after all. When I turned the system on, with the volume knob all the way down, I heard some crackling sound coming from the left tweeter. When I turned it up, with no music on, the crackling disappeared, but the fade-in and -out continued with the music playing.
The night the problem started, I apparently put a record's sleeve on the volume remote by accident and left the room to get a drink from the kitchen. With a delay, the volume began to go up. I ran back to the room, and a friend of mine who was in the room, grabbed the remote and turned the volume down, but the knob must've have gone up almost all the way up for a second or two. It is possible that it caused some damage to the tweeter. I'm surprised since we listened to several records with no issues after the remote incident before the problem revealed itself. But at this point, it's seems to be pointing to the speaker. The SR-17.5 from Silverline Audio are incredibly well-made and I know they can handle a lot (apparently max output is 126 dB), but I guess all it takes is a moment of inattention and bad luck sometimes.