Tones and rhythm are probably not enough if one doesn't grasp the significance of the notes being played. This seems to be a major stumbling block for those who do not like Jazz. It apparently sounds like a random jumble of unrelated notes, rather than a cogent expression utilizing a recognizable language. It's difficult to become addicted to something that one experiences as inherently off-putting.