Advice on whether speaker upgrade is in order

You have the amps that could make any speaker submit and play. That being said I bought some OHM 5000s in 2012 and my 4bsst did not drive them good enough. I traded up to the 14bsst and they become more dynamic. I remember a reviewer on the 7bsst said they would be better for rock. Or you can try to get a vintage Infinity Kappa 9. Notorious amp killer. By shutting an amp down because the impedence can go really low. I think your Brystons would show them who is boss

Room acoustics.

Do that first. :)

I would also ask you to clarify how your 803's are getting fed. Is the Velodyne crossover at speaker or line level?

Is there an EQ in your subwoofer chain?

Yes, yes - forgot to mention room acoustics.  I know that was high on the list, but I don't control the room (wife) so I will need to be creative on making any improvements.

On the feed, yes, I was unclear.  It is line level: preamp -> sub -> amps -> 803s.  No EQ added.

Thank you!

I’d sell the B&W’s and get a pair of Vandersteen Model 5's with integral bass amplifiers. There is a pair available now on eBay for $5995 from a seller in Brooklyn, NY. 

I'd keep all the Bryston gear - get rid of the subs! That "tinniness" is coming from the B&W tweeters!

You might  try running the 803s full range without using the sub's HF filter. Less circuitry in the audio chain is always a good thing.

As turnbrown stated,  I would try speakers without sub. Spend some time to hear if that improves midrange and highs. If you remain wanting more bass then look into getting a more full range pair of speakers having very smooth upper frequencies.  

What’s your budget?

get rid of the sub crossover- run the B&W speakers full frequency bandwidth.  opimize the speaker position for best bass- can they be brought from the wall 4-5 feet to avoid bass cancellations from the wall?  
then run it for a while to see how you like.....

What speaker cables are you using? 

Thanks everyone, I just did a little reading and learned about bass cancellation!  I have a little work to do on speaker placement and wall treatments before making any gear changes.  Thanks!

Why don't you listen to what the latest 803 sounds like. You're used to the B&W sound so anything else will probably sound off to you, although it wouldn't hurt to get out and listen to whats new from a few different companies.

First, look at GIK Acoustics. :) They have great looking things including art panels.  The soffit traps are also wonderful and can be stood up in corners.

I would suggest that you switch to a DSP controlled crossover with built in room EQ. Even without room treatment it can be a game changer.

I have no experience of the gear you have.
But I do have ATC SCM19v2 speakers, and they have changed my enjoyment hugely. So much more information, depth, and base (even though they are bookshelf speakers). 

I also can see what people mean about Bryston gear sounding a little tinny or bright at times.

If I were to evolve the system, would speakers be high on the list, and recommendations given this hardware?

So the one thing you know for sure is the Bryston amps sound tinny and bright, but the one thing you want to try is new speakers? Just want to make sure I understand the question.

The One Rule that has served me well over 50 years is always work on upgrading the weakest link. Your One Job is to find the weakest link. Could be anything from power cords to speaker cables to the cones under the components to the lowly fuses inside them. Yes even the room is fair game. Anything and everything.

Now out of all those things you could be looking to upgrade, there is one and only one component you take the time to tell us you know is flawed, and another one you take the time to tell us you don't think is flawed. So which one is it again you think you should be upgrading?

While you're mulling that one over, here's a freebie you can try that will probably help a lot.

Swap connections to run the B&W direct and full range. Do not use the sub crossover, except to adjust the sub. Right now you have your mains being driven by a signal you're sending through two extra God knows what interconnects, the assuredly crap crossover network in the subs, and then back to drive your speakers. Every one of those steps involves wires and circuits each one of which is adding a veil of grain and glare, which is also the problem with the Brystons. So you've unwittingly made a bad situation worse. 

Running the B&W full range will also improve bass response, probably by quite a bit, because the more bass drivers the better. 

But hey, don't take my word for it. Try it and see. Go and listen.

Thanks,will play around with getting the sub out of the way.  My recollection from years ago was that the bass was uninspiring, but this is an old memory, and without much attention to room treatments

In terms of my general inconsistency and speculation, my reading led me to hypothesize that speakers could be a weak link, and I had never heard B&Ws criticized for being too bright.  So I added my only sonic criticism of the system (a little bright), which is sometimes thrown at Bryston - just trying to provide info for the more knowledgeable to opine.  Who knew what you folks might say (e.g., yes, it will always be slightly bright, but you will get a huge lift out of xyz speaker....) :-)

Following and reading as this is interesting to me as I have old but very reliable 804Matrix fronts from early nineties I think and 805 N rear surrounds (sound anchor on both make dramatic positive impact to low end ) some unknown but ok sounding ceiling and in wall speakers that were in when I bought house.  The BW’s are my weakest links aside from wall ceiling believe but I’m not unhappy with overall sound.  Just upgraded a Krell HTS 5.1 avp to McIntosh MX 123.  Krell Kav 2  and 3 channel amps plus a sweet sounding MC275 mkV  powering fronts.  Kubala Sosna ics and speaker cable.  No sub yet until new speakers.   Thinking of maybe used Sony’s Faber Olympic III? Or more bw 803/4 D2/3.   

A few years ago my Plinius pre/power setup was sounding bright.
Did some research (before the internet had information and reviews as accessible as it is now) and bought cable that made a huge difference.
Was so effective I moved up their line by three models.

So although not ideal, careful cable choice can help tame a bright system.

Big question is are the current speakers meeting expectation. Is there a certain flavour no matter how much you tweek. Make sure your room and speaker placement is up to snuff first . Experiment and take your time listening. 

What your experiencing is the b&w’s overpowering your room. Major room treatment is required to tame these higher frequencies. 
My solution was to get rid of the b&w’s and two subwoofers. Purchased ‘full range’ Sonos Faber Amati Futuras. Now I have a very balanced sound with loads of very accurate bass. These speakers will play any type of music well. The wife will be satisfied since she’s looking at two Italian designed cabinets as opposed to the four cabinets you have now. 

I have Matrix 803 Series II.  My upgrade was subs.  It made a huge difference in the flexibility off controlling bass integration, and of course volume when needed.  I did time align the B&W speakers with the sub drivers, and use a separate amp and pre-amp for volume control and for power.