I have done this quite successfully. I'm not saying this is the only way, it is just the way I accomplished it.
First, I agree with the statement that a centre speaker can mess things up. I tested a decent one (Vandersteen) and it added absolutely nothing to what my main speakers could do as a phantom centre. Do your own test, though. Some main speakers don't do as well at creating that phantom image.
I picked up a decent AV preamp and a separate 5 channel power amp for the surrounds. The only commonality between my audio and video systems are that the main speakers are used for both purposes.
If I want to switch from audio to video, all I have to do is pull the speaker leads for the main speakers out of the one power amp and plug them into the other power amp, switch on the appropriate preamp and go.
I also have a pair of powered subs that are used only for video use.
It may seem like a lot of fuss, but it is the only way to accomplish really good sound in the audio system for me. There may be an AV preamp out there that sounds as good as my audio preamp/amp, but I haven't found one, and my audio system power amp at 70 wpc wouldn't run at video volumes easily.
Basically start with a good audio system and add on the pre/power and surrounds as a stand alone that takes about two minutes to switch to..