Every situation is different. I agree with all the posts but the culprit in anyone's system (including environment) can vary. Generally, for me, brightness is what causes fatigue and from my experience is normally due to a "hard" environment with few absorbtive materials (e.g., rugs, couches) or from a mismatch of speakers and amplifiers. Cabling can help alleviate that, but using cables as an equalizer may not provide the optimal solution (though it is cheaper than replacing speakers or amplifiers).
A bad source system is a nightmare. A bad cd is a nightmare. If either of those situations is there, forget about it. But, most source systems today that an audiophile would buy are pretty good.