I believe it's mainly the room.. and also possibly speaker placement. You can try toeing your speakers out slightly. Imaging may lose some focus but it should balance the HF response. You won't be able to effectively 'roll' equipment until you have these things right.
You may then also try DAC or streamer filtering settings, if it has them. This will be subtle, but can help somewhat with fatigue.
I did insert a tube preamp between my Class D monoblocks and DAC (which works great driving an old Aleph 3 direct, but my DAC driving the monoblocks direct with no preamp.. while detailed was quite fatiguing). The tube preamp worked wonders.. midrange bloomed, the soundstage gained depth and coherence.. and this made the system listenable for long sessions with very little fatigue. It took a lot of experimentation rolling expensive 6SN7's before I found the right ones however.
That said.. driving a Pass Aleph3 directly with the DAC sounded the best.. the most coherent and organic, and with the lowest noise floor.. none I could hear basically. My DAC's output stage was designed for this.
Back to the room, one thing I have also done (& my room is a disaster.. very asymmetric with boundary issues on the right and open to the left and behind), was to fabricate stepped speaker baffles out of lamb's wool panels to crop the dispersion of the midrange and tweeter and also mitigate cabinet reflections and diffraction. Basically the same thing Dunlavy used to do with his SC series speakers. It was quite effective at cancelling reflection issues and improved the sound stage immensely, and this perceptively gave the sound a more believable, more organic presentation. I could listen 'deeper' into recordings, suppose I could say. I've read these baffles should be made of actual lamb's wool, and not synthetic or felt. I used "1/4 panels for the base layer, and "3/8 for the top layer. This is great if you listen primarily in the sweet spot, but if you wander around the room while listening, large plants, curtains, rugs, etc, might be preferable.