@grislybutter Yes, confusing.
Normal sub cutoff is 60-100 hertz, your mains go down to 40, and the lowest your subs can cut is 80, maybe try 100? There maybe a variable lowpass XO out there for you, that way you can dial it in.
One sub can work just fine, usually you add another to balance it out, and fill in the bass holes one sub can create. A lot of that can be cured with sub placement. I have found with 2 subs, that you can great bass, but it can easily overpower the mains, becoming the focal point.
This entire setup is starting to feel like a compromise. As nothing is really working with each other. This has me thinking, does your amp, have a speaker A/B switch? Not sure why no one talked about it. You can just wire the subs to the B connections, then put the low pass filters on the A speaker, both with will get full power.