Hi Schuber,
I suppose it largely depends on what kind of records/music one wants to collect and the kind of money one wants to spend. If you are talking about searching for cheap used recordings souced locally, then yes, LPs are the way to go. In almost any city, one can find classical records selling for around $.50 to $1.00. This is a pretty cheap way to build a basic collection. Used CDs can also be found locally in most big cities, but, used CDs sell for a lot more. Of course both can be found on the internet, but, along with shipping cost and the inability to examine the records first, on-line shopping for LPs is not nearly as inexpensive and reliable as local shopping.
But, if one is looking for specific recordings or less common composers and repertoire, and one is willing to spend more, then CD might be the better alternative as far as availability.
As for sound quality, I think both CD and LP sound quality is so highly variable that I would not make any kind of generalization about which will sound better. Particularly when one is considering used records, the sound quality, in terms of noise and possible groove damage, makes it more likely that an average selection will sound not as good as an average used CD (although, as you suggest, LPs are cheap so that one can be happy keeping only a few out of a lot).