I suggest looking at it from a slightly different perspective-which some of you are already doing; where do the diminishing returns kick in more abruptly. I also am disregarding base-level set ups intentionally (base level skews the analsis).
So assuming a $5,000 and up budget/expenditure I believe they kick in most quickly with the phono stage, then cartridge, then turntable, and last with the tonearm.
I know many hear will disagree. The above is based on some degree of experience.