"I had made the assumption that 24/ 96 files would sound much better than 24/44.1 files but this was not the case."
Correct! It is not so much about the resolution but the provenance of that recording's original master. I have heard files in 16bit/44.1lHz that sounds simply amazing. For example, Shirley Horn's "Beautiful Love" or "If You Go" in 16bit/441kHz on Qobuz sounds stunning in my system.
You can take look at my "Digital" system (under my username) to see the steps I've taken to ensure optimal streaming quality vs downloads. The devils is in the details :-)