Are high sample rates making your music sound worse?

The notion that we may be overdoing it has always been at the back of my mind. 

I've found that up sampling to 96 kHz has never presented an issue but that above 96 kHz has and that includes recently recorded material.
A friend however gave me a CD of 'Ziggy Stardust' that was originally a MOFI JVC vinyl pressing and the higher the up sampling that I could go, the better.

@goofyfoot 



It stands to reason that if you upsample but have same bit rate as music was initially recorded at you would in essence kind of thin out the track altered.....losing possibly bass or bottom end...