I suspect this sentiment has already been expressed, but as a neophyte audiophile (meaning, I am learning an audiophile appreciation for sound, but may or may not ever have the resources to support the hobby), high end audio is like a foreign language.
At the Capital Audio Fest last weekend, I had limited time, and didn't recognize enough names to even guess which rooms would be best to visit. And saying that, I've grown up with a Dad who is into vintage audio, so I understand some part of the audiophile world, but the names I recognize are often of companies that are no longer around, from a time when no one would have dreamed of spending thousands (or even hundreds) of dollars on cable.
My impression of newer audio is that it is very hard to access. The friends I know who are well off are all that way because they work hard, and they try to be knowledgable in their hobbies. The entry hurdle for even speaking the language of high-end audio is simple too esoteric to make it a good value equation for most people with lots of disposable income.
And music is a time-consuming hobby in and of itself. Collecting and keeping track of high-quality source material, plus then making the time to listen to it, especially time to listen without distractions, is almost unthinkable.
B&O exists because it succeeds in providing an upmarket product that is visually appealing, fits in with high-end decor, and sounds better than Bose--at least in in-store demos. It's also in place in the right kinds of shopping areas, that cater to other high-dollar shopping habits. It also takes very little time to understand, set up or enjoy.
At some point with wealth, convenience seems to be the trump card, and high-end audio is not convenient and certainly not quickly accessible.