I think the hard part is not just composer and composition, but particular performance + recording sound quality. For example, DG had a superb roster, but I didn't like how most of their records sounded. Decca- often sublime and you can go down some rungs sometimes to London and even some of the budget series that were pressed in England. One, which I bought when I was a kid is the Martinon/Paris Conservatory of Divertissement / Jeux D'Enfants / Danse Macabre / Le Rouet D'Omphale which I first got as a Treasury series (budget) and many pressings later, I can still enjoy the cheaper copy. The Decca is now spendy.
I have quite a few EMI ASDs and their boxed sets from the same era of things like Shostakovitch, Vaughn Williams, Du Pre, which were all good. Some, highly regarded, are not as interesting to me musically.
The RCA doggies and Mercury Living Presence were hyped by TAS back in the day and finding clean copies is a challenge. I haven't checked audiophile reissue prices for some of these but would imagine they are spendy as well.
One of the issues I'd like to underscore is that if you talk to somebody who is knowledgeable (I have a few friends who really know repertoire), their focus is on the performance, not necessarily sound quality. Thus, if you dig into Scriabin, you'll see that many musicians think Vladimir Sofronitsky was one of his best interpreters (he also married Scriabin's daughter). But the recordings are a little rough.
Among the solo performances, the cello is one of my favorite instruments (no, I never studied cello) and of course Starker's rendition of the Kodaly piece for unaccompanied cello is killer. The early recordings on Period Records are mono-- I have a few copies, along with later reissues. I prefer the old copies.
The body of classical music is so large, even leaving aside modern or late 20th century composers that most of the university level folks have niche areas in which they go deep. Beyond that, it can be a very large sea to swim. Have fun-- many classical records are now depressed in value (though not the most desirable warhorses), so this is a good time to buy in my estimation. And classical records often didn't get trashed the way some pop/rock LPs did.
I'm hardly a guru. I did most of my classical LP buying when the LP was declared a dead letter, in the '80s. There are thousands of these records here that have gone unplayed by me for years, but I'm not selling any of them. The exploration is 1/2 the fun. Enjoy~