My first foray into high end audio had me originally lusting after a pair of B&W, they were aesthetically a good looking product with a great pedigree...it was the house monitor at Abby Road Studios, mind you. I was able to audition several models in my home paired with a McIntosh MC300, then with a borrowed Parasound. I felt every one I listened to with intent to purchase were immediately fatiguing to listen to given their very bright, in your face(in a bad way), presentation. My first pair of high(er) end speakers ended up being a pair of PSB Stratus Gold's for less than I was willing to pay for the B&W, and I was in audio nirvana...until that itch needed another scratch.
I still believe that whatever gear you decide on, the 804's will be your weak link. There are a pair of PSB Synchrony 1's on this site that would be an excellent foundation to begin your hunt for YOUR audio nirvana.