I thought that your suggestion was that there were reasons why both disciplines coincided. I was curious what those might be.
Black Americans is the answer. As their employment / career opportunities increased across the society, the number of them wanting to Box or be Jazz musicians decreased. There have probably been many potentially as good as Ali, Louis and Marciano etc ... they are just working in cooperate America. Or maybe in the NBA, NFL or MLB. All a lot easier than boxing or trying to earn a living in Jazz.
Other groups, Irish, Jews, Italians etc... traveled the same journey.
Cheers