Alot depends on our goals but you posting "but need to draw the line on cost somewhere!" is particularly relevant. Optimizing what we do so that it aligns with our sonic values for the $$ we wish to invest is intelligent. Each one of us can probably "afford" whatever we want within reason. I read alot on this site where someone has obviously overspent and we are wise to leave that kindof thing to someone else.
For me, I just don't want my hifi rig to be relevant in any way financially. That's me...other's may choose differently and that's their decision. We have a dedicated listening room but we also have the occasional car that drives past, the HVAC system cycles on/off and our golden retriever will occasionally bark or at least stop by to take a nap...and she snores....and like many here on Audiogon, she will occasionally pass gas.
I mention all of this because its wise that our investment in the gear should never be the end game, we should strive to maximize our enjoyment. The money is secondary. These are appliances that we can appreciate the job they do but they shouldn't be some shrine. In short, it isn't that we can't hear the difference between a Taiko and a Grimm...we can. Same goes for a Grimm and a Bluesound....is the gear going to serve the purpose we want it to serve or not...