I'll respond only for popular music and leave the classical to the more informed than me.
The most tragic and biggest loss is John Lennon, hands down.
But he had already made his mark before he died, and despite whatever wonderful music we missed out on because of this untimely death at age 40, his best work was surely behind him.
So from the standpoint of who we lost at the peak or upswing of their career, I'll go with Buddy Holly overall (though before I was born I'm aware of his impact). And from a musician standpoint, Duane Allman. So much brilliant music never got made...
