Well, as a huge fan of the run spanning Beggar's Banquet through Sticky Fingers, my reaction to Exiles was the same as yours. I WANTED to like it, I believed it when people so much smarter than me told me I was SUPPOSED to like it, but I just couldn't break into it. Over the years, I've owned 3 or 4 copies, all of which I played in depth, and then finally gave up and sold. There's obviously something there, but other than maybe 2 or 3 tracks, nothing to compare with the accomplishments of the preceding albums.
As Kavi says, to each his own.