Let me be perfectly frank and repeat what I have previously written numerous times before:
Whether your are listening to an ear-bleeding boom-box or a mega-dollar high-end system, they all sound pretty good after 3 glasses of wine - and listener fatigue will be a non-issue!
Honesty, listening fatigue came, as it used to be for me, from ''analyzing'' sounds (you know, the ''air'' around the instruments) instead of freely listening to the performance and enjoying the artist.
Once you free yourself of this, the listening fatigue factor is no factor at all. Easy to say, not easy to do, however.