surround processor?

Wasted a year, maybe two, chasing this one. Easily the worst thing you can do is add channels. The processor alone, merely having it connected nothing even going through it, however you want to call it, even that harms stereo quality. They are just the noisiest thing, much worse than a DAC, CDP or streamer. They cost money that takes away from where it could actually do some good. Extra speakers, amps and cables means only being able to afford worse ones for where it matters, the stereo.  

That is why after two years banging my head against marketing and the CW I finally got up the courage to buck the trend and do it right. https://systems.audiogon.com/systems/8367 I wish you well.

If 2-channel is very important to you, IMHO it’s best to use a dedicated stereo preamp or integrated and hook the front L/R preouts of the AVP into it. 

soix, I assume in the setup you describe the AVP is out of the loop for stereo, but its L/R preouts go to by-pass inputs of the preamp for surround. That’s the setup I use. HDMI input to a Bryston SP3, balanced analog from the SP3 to an Ayre preamp. Balanced analog stereo inputs go directly to the preamp, so the SP3 is out of the loop.

Best to have two separate systems, sharing only some sources if desired.
Then you can enjoy surround AND the best of stereo that you can muster.