Recently, I faced the same dilemma. I have a very good system. However, it is in my nature to always seek "better", even though by now I have taken mine beyond what most people on this forum have proposed as modest to major potential improvements to the original poster's system. For example, the concrete floor of my AV room has been sawed to be physically separated from the rest of my house, then covered with acoustical matting and finally with carpeting; my walls and ceiling are of multilayer sheet rock; acoustical paneling was placed on the walls wherever necessary to minimize undesired reflections; and the room/system itself has been tuned via Dirac Live. Not surprisingly, I have continuously upgraded my equipment (including power conditioning and cabling) for the past 15 years.
Increasingly, I have become aware that (i) the law of diminishing returns is a fact and (ii) the more complex one's audio system, the more interrelated -- and therefore sensitive to even what would seem to be a very modest adjustment -- is its performance, giving rise to the prospect of unintended consequences. My final education, if I can call it that, came last month when I supposedly upgraded a power cord for the Ethernet switch in my system. The result of this was a clearly demonstrable, and significant, degradation of the magnificent sound I had been enjoying.
From this experience, I now am trying to listen exclusively to the music playing -- both to its content and how it sounds and to stop trying to analyze whether the reproduction is sufficiently "life-like". This has been hard for me to do, and yet I intellectually recognize that, given my 73 year old ears, my system is already as good as I can hear.