How do you know when to replace a tube?

I'm new to tubes. I have a Baby Sophia powering a pair of Klipsch Quartets from an AR ES-1 turntable. The sound is often warm and lovely but sometimes thin and metallic to the point of making me want to stop listening. Sometimes the first couple of songs on an album sound soft and sunny but the last songs sound flat and brassy. The tubes on this amp were the original tubes used by a previous owner; I've used them maybe an additional 50 hours total. Does the sound quality of a tubed amp gradually degrade or do the tubes simply stop working? Will certain types of music reveal a degraded tube or will it be pretty obvious with all kinds of music? Thanks for your responses. I want to get back to all warm and sunny sounds!
See if you get the same results with a different source like a CD player. If the same thing happens with CD's then it may be tubes. Tubes can change with temperature and not always for the better. However, when you own tube equipment it is always a good idea to have spare tubes on hand.
The other thing that comes to mind is your AC power. If everything else checks out ok you may want to try a power conditioner.
Change in sound between parts of a record side have nothing to do with the tubes--that is either something in the recording itself or a problem with cartridge alignment.

Unless you have access to a really good tube tester, the best way to determine when any tube needs replacement is to get a complete set of new replacement tubes as back up. Once in a while, replace the tubes with the new set to see if the sound has changed signficantly. Don't do all the tubes at the same time because that won't tell you which are beginning to go bad. I would start with the power output tubes first (they tend to have the shortest life), then try the small signal tubes. If the sound comes to life with a replacement, you know that the replaced tube(s) have started to go weak. Sometimes, older, weaker tubes also start to become noisy; if a good cleaning of the tube pins does not cure noise, it is probably time to replace the tube.
This is a bit off-topic, but not by much. I have a Sophia Baby that I have upgraded all the capacitors on (I just bought the ones Richard at Sophia recommended). I also bought a pair of Western Electric 396a output tubes. With these few upgrades, your Baby will sound much, much, better, you will be very glad you did them. You can do this yourself, as I did, or send it to Sophia, and they will do it for you. The WE tubes, NOS, are very expensive, so I found a good used pair for much less money. I have been using this amp for about a year and a half, with no problems.

This may or not be of interest to you, but it's something to think about, if you are going to re-tube.

Best of luck to you,
I'd first check the tubes biases whether all are still holding up to minimum factory recommended. Then also your AC voltage, fluctuations/instability of which could lead to the change in sound you described--hence, a stabilizer may be all you need.