@alexatpos - I can offer a partial explanation in the States and that is the explosive growth of the music business as a youth-aimed product in the later ’60s, when record companies realized that all those kids who liked the ’new’ music were a huge market. That younger market had seen a shift from the big band era before WWII to the rock ’n roll (think early Elvis, Jerry Lee Lewis) era in the ’50s, the first British invasion a la the Beatles and Stones and then, after Monterrey Pop, a whole range of sounds, bands, records and "life style" to be sold.
Jazz had no real market in this environment, but what is interesting to me as we entered the ’70s is the re-emergence of some serious players in spiritual jazz scene, on private and small labels, merging jazz with Afro-centric, black power stuff that delved into funk, polyrhythms, eastern influences, gospel and soul. It wasn’t straight ahead jazz, but something very different, and had a socio-political aspect that spoke to the black experience in America during a period of social upheaval and raised consciousness in the communities. This stuff rekindled my interest in jazz in the last few years as a departure from straight ahead styles; it was, with few exceptions, not very well known or sold beyond the local communities where it grew-- Horace Tapscott in LA on Nimbus West, Tribe Records in Detroit, Strata East in New Jersey/New York. Some brilliant stuff, executed by some very well known players who turned inward when it became obvious that jazz was not a mainstream genre. Well worth exploring if you haven’t; I have found some great records from this era- Marchin’ On by The Heath Brothers, Earth Blossom by the John Betsch Society, and of course, Gil Scott Heron’s Winter in America, a sort of soulful lament of spoken word and Fender Rhodes.
I can listen to this and so-called proto-metal (very heavy rock that anticipates the later heavy metal scene, without the cookie monster vocals or guitar shredding) and enjoy it all for what it offers from that era.