Have I Hit The Point Of Diminishing Returns?

No, you have not hit he point of diminishing returns, not even close! There are many, many levels/steps of quality and dozens remain above where you are now.

I am not saying this to belittle your system; you have great gear and I'm sure it is great sounding. However, the steps of improvement are literally endless. In my system also I do not see the finish line, that is, at a level which cannot be improved. Regularly I have dramatic improvements and have for years.

I strongly disagree with the perspective that you must spend multiples more to achieve a 20% change in sound quality. I would nearly reverse it, and suggest that in trying many different components you can achieve a seeming "100% improvement" in not all that much increase in cost, perhaps twice what you would get for the sale of any given component. i.e. sell for $1K, add $2K = $3K new component. And that is potentially at the upside of what you might have to spend. This is because there is a HUGE variance in component quality from manufacturer to manufacturer.

Example: Some of the new DACs are mind-blowing in terms of improvement of digital quality sound, and you can have these for nowhere near nosebleed prices. The new DSD capable DACs are "affordable" as considered to be truly High End, where some components are priced by the multiple thousands.

I encourage you to do some shopping/listening and experiment with your rig. My experience is that the "point of diminishing returns" is perceptual, not actual. And, yes, changing cables, for instance, will alter the rigs sound most likely in a profound way.

I don't think it has that much to do with money, but it certainly could end up that way. You are in a position now where you have your system tuned you your tastes and you enjoy it. I wouldn't think in terms of "better", but instead, think of "what would I have to do to get more enjoyment from my system". IMO, you would need to home demo components and cables individually to see if they really make you want to listen more or less. A supposedly better component might upset the balance of your system and diminish the listening experience for you. It would also be an idea to speak with a good dealer and let him make some recommendation based on your taste and existing system. A good dealer can do this - a box mover is just going to try to sell you what they want to move, so be selective. Good luck.

You know that it has been written many times before that the latest gear is not necessarily the better sounding gear - most often it's just different sounding gear & at other times it's worse sounding.
You electronics is all good stuff. I'm not a fan of B&W anymore but if you like the sonics then I'd suggest keeping it.
The thing about B&W is that it needs a lot of current to make those speakers come alive. So, from my personal B&W experience, I'm going to say that you are perceiving hitting a ceiling with this gear simply because you do not have sufficient power for the N804. If you can manage it budget-wise I would recommend getting a high current amp - even a class-D power amp. Something with outrageous output power - I'm thinking 400W/ch.
I was never a fan of Cardas cables - too rolled off for me. I believe that this is not helping either. Several inexpensive but supposedly very good sounding brands to try such as Morrow Audio, Signal Cable.
FWIW.

I wouldn't so lightly dismiss Cardas as "rolled off", as if that's a bad thing. From what I see, many, if not most of the complaints here concern bright or forward systems. Many audiophiles fall into the same trap - they construct these "accurate" systems and then bitch about the brightness. It's a delicate balancing act. The room has a lot to do with it as well. If you're in a big room and sitting pretty far from the speakers, a more neutral "better highs" cable may be the thing to balance out the degree of softening the room may have. But if you're sitting 8 feet away from the speakers, those cables/components may tear your head off. This is not a simple thing and you have to let your ears be the judge. There's an old saying - the more you open the window, the more garbage flies in. That's why I'm saying that upgrading has to be done carefully and preferably with the help of someone who knows the gear and whose ears you trust.

Interesting views. My power amp is 225 wpc dual mono, so I THINK it's powerful enough, but I could be wrong. Love how the B&Ws and Cardas sound in this system, although I agree that they may not work in some other systems. Essentially, there seem to be two opposing points of view: (a) marginally "different" sound can be had for a significant price, but it may not be "better" sound or worth the money; and (b) it is possible to achieve 100% improved sound for a reasonable cost, but is component/system synergy dependent. Confusing, no doubt. BTW, my new listening room is a sucky 12x12x8, but I have treated it with GIK Acoustics products which have helped tremendously.