I can completely relate @garebear!
The album you refer to is The Band’s second, commonly referred to as the brown album (it’s self-titled). Their debut---Music From Big Pink---was released a year earlier, and caused quite a ruckus (Clapton disbanded Cream after hearing it, traveled to the Big Pink house and hung around for a coupla weeks, hoping and waiting for them to ask him to join. He finally realized they didn’t need or want him 😉).
My crowd was comprised mostly of musicians, and Music From Big Pink divided that crowd into two camps---those who got The Band, and those who didn’t. I was amongst the latter, and try as I might I just could not understand why the smartest guys I knew loved that album. I was into all the usual stuff---Cream, Hendrix, The Who and the rest of the power trios (The Who were a quartet, by Daltry isn’t a musician), as well as the doors, Buffalo Springfield, Moby Grape, The Byrds, The Beatles (to a degree), etc., and MFPB sounded very foreign to me. I couldn’t in the least relate to it. And that bothered me.
Then in the spring of 1969 my band got a gig opening for The New Buffalo Springfield (drummer/singer Dewey Martin the sole original member remaining) at a local San Jose high school. We played our set, and I carefully watched and listened as TNBS started theirs. As the set developed, I became confused. None of the members seemed to be playing much, but they sounded SO good. At about the halfway point in their set, I had a sudden and overwhelming epiphany: Oh, NOW I get it! The "it" is the musical style called ensemble playing: Playing in service to the song, and in relation to what every other member is playing and/or singing.
That experience changed my musical life, and when the brown album came out later in ’69 I was ready for it. I and every musician I knew studied that album from front-to-back, for hours, and hours, and hours. I had to complete relearn how to play drums, and that album (as well a MFBP) is the blueprint for how to play musically.
Music From Big Pink and the S/T album are without question my two favorite album of all time, by a wide margin. So musically rich, so deep. Expert musicianship, three fantastic lead singers, and great songs. That's what it's all about!