Good stores in LA

Gene Rubin is great If you are Into Naim electronics and Harbeth speakers.

Down here is SanDiego, which until reading the above posts, I'd have thought of as the end up the world, has a couple of places of note:
1, just up the street in Carlsbad has Mac, CA, PassLabs, Paradigm speakers and a bunch of other goodies. I bought a PSAudio Soloist outlet there last year.
Down South is another operation with no less than 7 or 8 demo rooms. Wadia, Krell....electronics and speakers.....B&W, Rotel, Naim, Rega, Magnepan, and many more. Good used selection and great staff.
There is a mid-fi place or 2 with NAD offerings, include the 'M' stuff. More boutique AV/HT places than you can shake a stick at finish it off.
Not a lot, but overall pretty nice.

Check out Digital Ear in Tustin, CA, they have the top shelf gear you're after:

http://digitalear.com/

Stereo Design in San Diego also has top stuff:

http://www.stereodesign.com/

Sunny Components in West Covina, CA has good stuff, but the owner is very difficult to deal with and he's by appt only, so the hassle may not be worth it.

http://www.sunnycomp.com/

Jay at Audio Revelation (San Diego) and David at Weinhart Designs (Beverly Hills) carry Magico and have good reputations, but they are by appt only.

Brooks Berdan is probably worth a visit, and from the comments above, they seem to be one of the more approachable dealers around.

Upscale Audio in Upland, CA have a great reputation among tube enthusiasts, but I don't think they do walk ins or appts really...

Brace yourself for some serious LA HiFi attitude, most times it's not pleasant dealing with dealers, but a necessary evil if you want to enjoy and explore this hobby...

Weinhart Designs also has the new Magico M5s. Listening to the M5s alone would be worth a trip to LA...