Ralph says LPs are best and another party says tape is best and I pretty much agree with both statements.
How? If the tape is absolute top quality it will win. However, even "master" tape from sources like Tape Project can get walloped by LP, at least in my system.
One tape I have is undeniably the best, a true original safety from an artist done in his recording studio. This was back up tape for a recording session and so untouched, it is primitive hard left and hard right 2 channel (no mix down) and never duplicated. I'm guessing based on the music it was recorded on an Ampex 350. This tape kills every source, regardless and by a mile.
However, tapes from other good sources that are "master dubs" do not compare to this safety and sometimes not even equal to the best possible LP.
How might this happen? I suspect the true original tape is seldom pulled for anyone. A copy of this master is at hand and that is duplicated when required and sent out.
Now this user gets a "master" that's two or three generations away from the original and even Tape Project frequently dubs again to 1/2" and then that is copied over to 1/4" for subscribers.
Look where we are now compared to true master.
A friend of mine in the record business said that a digital master file at the studio was absolutely amazing and in some ways better than analog. However, the first time it's moved, transferred or copied something happens to it and it's never the same (his words).
So the answer to what is best quality depends. Tape potentially is supreme but most of us do not have access to even one of these at best example, much less a usable library.
As for why not tape instead of vinyl records, I think most points have been stated accurately. My comments are based on listening with my equipment and perhaps a better tape machine could change the outcome.
However, for me to spend more than I already have on a tape machine would be foolish. My Studer A810 with full restoration is already my most expensive source when divided by number of software titles. Basically, my LP library is the most precious thing in my system.