S/PDIF is about the worse possible platform for carrying digital data.
This is your experience with your particular equipment, not something that you can make broad conclusions from. There is nothing fundamentally flawed with S/PDIF as a transmission medium. There is no reason why it cannot be as good or better than any others, except perhaps lack of error correction. It is double-terminated, so if the interfaces are properly designed, it will have superb signal integrity and very low jitter. I routinely get ~15psec at the end of the cable across 75 ohm terminator.
This is the rub. Very few designers get the driver right. Wrong impedance and wrong voltage. I used to mod a lot of CD players/transports and computer audio equipment. Not one of them was designed right, even from Sony.
Steve N.
Empirical Audio