I think differences between 24/96 & 192 would have to be very slight.
That would be my instinct as well, and I would expect that differences in implementation and quality would generally overshadow the difference in sample rate.
Compared to redbook, though, although most of us are aware that 24 bits represents a 256-fold increase in information compared to 16 bits (at least potentially), I would view the benefit of the higher sample rate (even at 96kHz) to be at least as significant, if not more so. It is probably tempting to think of it as a little more than a doubling, but I think it is best viewed in relation to the Nyquist frequency (the 40kHz minimum sampling frequency which is theoretically required to capture a 20kHz bandwidth). Redbook's 44.1kHz exceeds the Nyquist rate by about 10% (which to me has always made it seem wondrous that it works as well as it does); 96kHz exceeds it by 140%, which should make possible vastly reduced side effects from anti-aliasing and reconstruction filters.
Regards,
-- Al