Listener fatigue also comes from digital artifacts such as jitter and bad phase relationship, and from tweeters that suffer from excessive overshoot, ringing, and "oilcan" resonances. It can also come from a persistent strong "one note" resonance, most commonly in the 100-200 Hz region.
There are many contributors to listener fatigue, but a little room ambience and reverb is pretty far down the list. When you consider that indoor live music is largely defined by the reverberant field, and that some of the more reverberant concernt venues are highly prized (e.g., Boston's Symphony Hall, Cincinnati's Music Hall, etc.) then the idea that room interaction is the primary cause of listener fatigue borders on the preposterous.
Add to this that dipoles such as the Nolas, MarinLogans, and Magnepans all rely on the backwave, as well as highly regarded bipolar and omni speakers such as the Mirages, Ohms, Shahinians, and MBLs, and it's obvious that--while bad room interactions can contribute to listner fatigue--there are plenty of examples of speakers that rely on beneficial room interaction with very pleasurable results.
Note also that in the latest iteration of Sonus Faber speakers, (which as a group are regarded as *very* fatigue-free), their top line $200K "The Sonus Faber" and $120K Aida models feature front and rear-firing drivers for a bipolar radiating pattern. This obviously brings the room reverberation into the playback presentation.
Taking room interactions out of the radiating pattern doesn't remove listener fatigue; it makes it easier to hear more precisely what's on the recording like in studio monitoring, which may or may not be fatiguing due to other factors.