I cannot believe nobody has even mentioned the live "Ella in Berlin." It contains a famous version of "Mack the Knife" where Ella forgets the lyrics and makes up new ones on the spot, and a death-defying scat on "How High the Moon." (Contrary to Easy_e, I love to hear her scatting.) Sound is not great and variable on the recent "complete" version, because it includes several tracks from less than pristine sources.
If you prefer the more intimate setting, you should be a bit careful with the Songbooks (I have all of them) which contain primarily big band arrangements.
Along with "Pure Ella", there was an album called "The Intimate Ella" which was also just voice and piano.
The first two Louis Armstrong collaborations are great. I've actually never heard the Porgy and Bess, but I think it has an orchestra (not sure). My favorite of the Joe Pass collaborations is the second, "Fitzgerald and Pass Again." By then (early/mid 70's) Ella's voice had begun a decline, but she made up for it with greater attention to lyrics.
By the way, I had trouble getting a clean vinyl pressing of the Pass/Fitzgerald album when it came out, and I wrote a letter complaining to Norman Granz. He sent back a handwritten apology and six copies of the album to make sure I found a good one.
That's class.