Transients hurting my ears on hifi solid state amps

It is highly likely that power level is your problem. Even at low volumes, moderately efficient speakers can benefit from higher power and it can significantly reduce fatigue.

I recommend looking into separates. Pre-owned Parasound A21 amps can be found for ~$1200. Their distortion profile is primarily 2nd order, which is very rare among amps below $5K, and output enough power to provide plenty of headroom for nearly any speaker. Pair an A21 with a low distortion preamp like the Topping Pre90 and you’ll have a very smooth and low-fatigue setup that can takes up little more real estate than an integrated.

Another thing: if you’re one who typically indulges in adult beverages during your listening sessions, practice cessation for a few days, and try listening earlier in the day when you’re alert and rested. Listening while already fatigued just accelerates listening fatigue IME. Alcohol tends to have that effect as well.

I remember learning that our ears are most sensitive in the frequency range where we are experiencing hearing loss. Apparently it's common for the high pitched voices of children to bother some grandparents. ;-) 

Thanks @holmz for the various very good points and suggestions. Just a clarification on one point, as this is one of the things I'm trying to understand here:

I doubt that the amps are any “faster”, as most amps play 20kHz just fine, and nothing is going to arrive faster from a 2kHz signal, than a 20kHz signal.

With fast I mean that the transients, the typical example being the hit on a snare, are quick and powerful. So in a graph measuring the sound pressure, the peak would be higher and narrower on a "fast" amplifier and lower and flatter on a "slow" amp. The sound is more compressed in a "slow" amplifier and thus the hit of a snare or the initial transient of a piano note will be felt as softer by the ear.

I understand that there are other factors discussed that heavily influence the perception, but I do believe that this is part of the equation. Low powered amplifiers would in my understanding be therefore more suitable to avoid the problems I've had. It is not so much sustained notes that cause issues for me, but the percussive sounds and initial attacks on many instruments.

Pre-owned Parasound A21 amps can be found for ~$1200. Their distortion profile is primarily 2nd order, which is very rare among amps below $5K, and output enough power to provide plenty of headroom for nearly any speaker.

Your argument is exactly the opposite of what I'm discussing in my previous post and I find that really interesting. I would need to experience that for myself and see whether the distortion profile would be enough of a key factor. I definitely get the idea of more clarity making it easier to hear details and thus make listening less fatiguing. I wonder whether this is what I'm sensitive to. The combo you propose does sound very attractive and reasonably priced.

 I misquoted the power on the Pass though.  It’s 25 Wpc at 8 Ohms, 50 Wpc at 4.

Thanks for the clarification @wlutke, 50 did sound a lot from what I remember about those Pass amps. My speakers are not the most efficient either, so not sure it would be a great match, except if my low power hypothesis is correct. I don't mind if the sound is smoothed out a little, like on my tube amp in triode mode.