Would you move to a bigger house if you could just to have a better listening room ?

Keep the house and dedicated listening room, ditch the shepherds and labrador, renegotiate services rendered by the wife or ditch that too--that is the way to downsize.

I have had a dedicated listening room in most of the houses I've owned, and those rooms have been larger than the one I have now, which is a spare bedroom, but I find that I enjoy this one just as much.

I have considered adding on, but building is so expensive now that it is easy to just be content to love the one you're with.

In a heartbeat.  

When my wife and I moved from Nashville, TN to Charleston, SC in the Summer of 2021, one major deciding factor in the home we ultimately chose was indeed the 22"X25", dedicated listening room. It even has it's own private full bath and HVAC register.. It also helped that it is on the 6th tee box of our golf course and has a deck completely across the back. My wife got the pergola she always wanted.