Wiring question : amp-sub-speakers


I am pretty new to audio world and am looking for an advice regarding wiring Denon PMA 2000 IVR integrated amp, Polk PSW 550 sub and JBL studio 530 speakers. Current wiring is, Amp to JBL using the main binding post and Amp to Sub using preout. It sounds great up to certain volume level and JBL 5" woofer starts over-excursion, aka bottoming out, at a higher volume(knob 11 o’clock & >90 decibel) to fast tempo low frequency sound like drum kicks. I am not sure how to improve this situation. Sub manual suggest "Speaker-level" connections, i.e., Amp to Sub and Sub to JBL. Is it connection issue, JBL issue, or Amp issue? Thanks in advance.
ubee67
I am not sure about  Polk but I had a NHT sub years ago that only sounded right if wired amp to sub to main speakers and you set the crossover on the sub. 
ubee, your problem is that you are not using a high pass filter in front of your amplifier. You need to get the bass out of your main speakers. The PSW 550 does not have a high pass filter in it's cross over. I believe your integrated has a preamp output and a direct amp input. You could use an outboard cross over in a loop using the preamp out/amp input. You would then disable the crossover in the subwoofer. The only other way to deal with this would be to put an appropriately sized capacitor in series with the woofers in your main speakers to roll them off at 6 dB/oct. 
I am not sure how to improve this situation. 
Easy: new speakers. Your bookshelf speakers are max'd out. 

Your amp has tone controls so what you might try short term is turn the bass down. Your little speakers can't handle it. Then turn the sub up to compensate. 

Its a crap solution but all there is without better speakers, since fundamentally your problem is nothing to do with the speakers its you. Play them within the range they are intended for, or buy ones designed to play within the range you want.