I'm also not a religious person. But I don't have a problem with religious music.
The Pärt "Te Deum" is probably my favorite piece of music from any genre from the past year or so. When it was new to me, I listened to it every day for weeks; now, a year later, I'm still listening to it at least once a week. The version on ECM is extremely well recorded, too, and well performed. It's about half an hour long. From his "Tintinabuli" period, so neo-medieval: the voices are accompanied by a digital recording of an Aeolian harp (!) and a prepared piano (!!).
But I forgot to mention Haydn's "Die Schöpfung" ("The Creation"), in the performance by Gardiner on Archiv. That is one of my "reference recordings": the orchestra, chorus and solo voices are brilliantly compelling in every way. There's a version of this composition in English, too, also performed on original instruments and conducted by Hogwood that is very fine. Haydn himself revised the score to work with the different cadences of English words. But the German in Gardiner's version will never be bettered.