Is it all worth it?

Yeah, I'm in the process of selling all my "high-end" equipment.

After 35+ years in the hobby and endless cycles of buy...buy...buy...then dump it all, I've finally come to understand how my mind works in this regard. The more money you spend the higher the level of expectation. Beyond a certain financial point, one arrives at a system where it isn't the music drawing the focus of attention during a listening session, it's the equipment. At that point the battle is lost and it's time to start over. 
When I sit in the little TV room listening to my secondary rig, it's all about the music. I wish I could just be satisfied at that level but my OCD won't allow it.

My salvation came in the form of variety. When a person is endlessly pursuing "the One," it's a good clue that they won't find it. Very likely there is an insatiable drive to hear something different, no matter how good the rig. Buying and dumping won't address the need. The need is to enjoy the variety and gear as much as the music, and there is nothing wrong with that. It's a fabulous way to enjoy the hobby. 

The solution is simple; divvy up the budget to get the best components or speakers to express variety, and watch how satisfaction blossoms. Bored? Simply change some cables, a component or speakers. If one has the space and the means but is not willing to spend the money on this, then there is no justification for complaint, especially when at all price levels this diversification can happen, even with inexpensive speakers such as the A40. 

For instance, something as simple as having a pair of more affordable panel speakers on hand to swap out with the main speakers can keep much of the OCD-like dissatisfaction away.  Or, if on a severely restricted budget, find several garage sale or thrift store speakers and rotate them. The variety of experience is lovely, and it doesn't need to be costly. 

"Very likely there is an insatiable drive to hear something different, no matter how good the rig."

I agree. Variety is the spice of life, right?

I think explains most of what drives able bodied audiophiles to change over time.  Its a basic force of nature that drives people and explains a lot.

Can anyone honestly say they have ever been able to maintain a constant high state of audiophile bliss day in and day out for an extended period of time with just one system? Sometimes you just want something a little different. then you are ready to go back to what’s really the best to you.

I keep my reference system and various others around as much as I can in order to best deal with that.

A much better approach than continuous change and upgrade if one can accommodate it.