Would you move to a bigger house if you could just to have a better listening room ?

Not a chance. The waste of extra property tax, heating and cooling, and maintenance would eat away at my peace of mind. In fact, I'm building a new house soon and it'll be in the 1800 sq. ft. vicinity. One good thing about living in areas that allow basements is that it is the ideal place for a listening room. The concrete walls are natural bass insulators and it's fairly inexpensive to make an appropriate-sized listening room.

I moved to a smaller house and got a dedicated listening room. 
 

theaudioatticvinylsundays.com

@unreceivedogma I did basically the same, from a 25X23X11.5 dedicated less expensive room to a very expensive, near SOTA 20X15X10 room where it was extremely easy to locate speakers correctly without DSP.  It has 16-17" thick multi-layered and sealed walls, a 12" 3000 psi steel reinforced concrete floor and built-in bass traps using activated carbon filtering chambered boxes throughout the walls.

@hikerneil stated " If you had this new house, and could design your own perfect 2-channel music room - how would you size it? "

There is a few Thousand Years of History where the human has developed disciplines for controlling sound. This has evolved continuously, as a result of careful study of the most efficient methodologies. 

Amphitheatres go back to BC, which one could call the earliest Audio Visual sound system 😎.

Auditoriums arrived much later and will usually be seen as an enclosed space, die to the Roof Structure. I would take a Punt, and say this is when controlling reflected sound become a new area of science, as the musical experiences to be had in a auditorium, would have been available to quite well off educated individuals who were paying good money for the attendance at the location being suggested as being the ultimate of acoustic controlled environments??  

Today there are University Departments that have done extensive studies, and shown the merits of what can be referred to as Traditional Room Dimension Ratios, in comparison to modern conceptions for a Room Dimension Ratio.

Additionally, there is enough known through research about how materials used for a structure and the structure itself will impact on the sound, i.e Solid Concrete Floors, Suspended Timber Floors, Solid Brick / Blockwork Wall, Studded Dry-Board Walls and Density of Cavity Insulation plus Insulation Density itself.

Much of this is Science and not the easiest to break down into layman terms, but fortunately due to the popularity of the Subject, as a result of Recording Studio Structures and Home Entertainment Spaces along with Music Replay Interests, there is plenty of basic description to be discovered.

My own room, which is to be a outside of the home room, is now recently changed with altered ratio dimensions, as a result of a new discovery of work undertaken and is to be produced using dimensions produced by Richard Cox from investigative works carried out at the University of Salford.

With the want to increase the floor space in the main home, the Main Structure is increased in size, with the intention to add to the spaces suitability to be multitask. With a consideration for my Wife, the main homes space for a Gym is now relocated to the New Design Structure where a Gym /Sauna is to be put in place at the North End. Abutting the Gym Room will  be the Audio Equipment Space, that will be created to be a very secure storage, of a dimension easily able to be kept at a suitable temperature and humidity.

The Wall that separates the Main listening Space to the Audio Storage, as a resuly of speaking to friend who designs/builds speakers is to be air sealed cavity of a yet to be determined Volume. This will allow for a Insertable Panel to be used to allow for different drivers to be tried out as a Speaker. This could work out very well for Audio Purposes as an alternate Speaker to ESL's, but will certainly be quite useful for Audio Visual Movies and the like.

The main space outside of the Secure Space for Audio Equipment will be created using dimension ratio's from Richard Cox, once complete it will be a Blank Canvas to undergo treatments to create the most attractive end sound that can be achieved for Stereo Replays of recordings. Audio Visual will be the poorer cousin in relation to creating the ideal end sound.

Recent impression made through experiencing different Speaker designs has brought to the forefront a new desire for the long term used Audio System to be introduced to a wide selection of Speakers from a range of designs such as my  ESL's being used for their first time with Subwoofers and possibly a Upper Frequency Driver as well? vs Horns vs Cabinet Types and even a DML design. Having a space such as this, will take a little getting used to, as the Audio Equipment has always been nurtured within the home.

The Space will lend itself much better than the home space to having such a speaker collection at hand and will be more reassuring in relation to the Valve Equipment no longer being within the home, especially the 845 Monoblocks, with the risks these and other Valve devices pose as a Audio Device. 

My wife will also benefit, as a listenable throughout the home SS based audio system will be put in place, to give her a musical experience as she would like to regularly experience.      

We moved into a newly-built home 3 1/2 years ago and the developer was willing to somewhat "customize" the bonus room above the garage, resulting in the best sounding room I've ever enjoyed!  This house will be an awesome place to grow old in!  So "no", would not be interested in a larger home!