Do caps deteriorate with time even if an amp is not being used?

Electrolytic caps loose rated breakdown voltage and also dry out.  They loose rated breakdown voltage because electrolyte "eats out" dielectric - aluminum oxide (presence of voltage rebuilds this layer).  They dry out - especially when hot.  Each 10 degC temperature increase cuts life of electrolytic cap by half.  If you store gear in cool place caps should be good for 50 years, except for the breakdown voltage problem - that can be fixed by introducing voltage slowly.  SMPS power supplies might not make it possible since some of them don't operate below certain supply voltage. 

I believe it is very dependent on what the amp is and the quality of components that were used. If it is mass-market junk then 10 years may be a big deal. If it is high quality equipment with good components, then 10 years should be nothing.

^^^^^^^^^^^
not an amp but still capacitors

I have a Studer tape machine that failed on me a few years ago. It would power up, but when you spooled up a tape and hit rewind, ........nada.

My tech told me "it’s probably the power supply capacitors. I have them in stock bring it over". As I waited and watched in amazement, if you have ever seen the inside of one of these machines; he checked out the power supply capacitors, and they were showing/testing "like new" . Huge things and I remember Made in West Germany labels.

The new, replacement ones were sitting on the table. So do we switch them out I asked. He said "No, I wouldn’t touch those."

Lack of use and lack of cycling caps are their worst enemies. Particularly if the electrolitycs are of "exotic" type.