I do not earn a huge income by any stretch of the imagination; however, I am moderately comfortable. I think the first thing to affording this hobby is to live life financially smart. I don't live in debt. I only use credit cards for making on-line purchases and only when I have the available funds which allow me to pay the entire balance immediately. I only pay cash for cars. I put aside money every pay period for general savings, general investing, retirement and both of my kids college funds. I own a decent home (about 3,400 sf). My annual income is about $100K. In a typical month, I have 5 bills to pay - power, internet/Comcrap, mortgage (includes ins. & taxes), kids school lunch program and phone bill. Of course I have regular living expenses - groceries, gas, spending money, etc. . .
My kids and I take a few nicer vacations each year, usually out west skiing or to Florida. A Canadian fishing trip with my son and several smaller vacations.
I have owned some pretty good mid/upper-tier equipment (nothing in the stratospher). Krell FPB amps, Levinson, BAT, CJ, Pass amps. Some great digital equipment - DCS, EMM Labs, Esoteric, Wadia, Levinson, Audio Aero, etc. . . Some fine speaker systems (Wilson, VS, B&W, Gemme, Totem, etc. . . ).
I think the key is to live responsibly, always saving and only spending when doing so does not present a potential hardship down the road. Living in debt is what kills most people and takes away opportunities. Of course a very high income (which I am sure most people of this site have - many times mine - helps).
To the original poster. I would learn to live with a lesser quality "out-loud system" and spending a little money on a great headphone system may be the best way to go. One can get a great pair of headphones for a few hundred dollars that can outperform some of the best speakers made! Secondly, go with mid level cables that are a few generations old. I just fail to recognize (via listening) that the latest greatest $3,000 speaker cable of today is really notably better than the $500 (used price) two generation old cable that they replaced.