Would it not depend on the frequency response of your speakers in your listening room with your tastes for sound in mind?
No. The reason the sound changes is due to distortion- LOMC cartridges and the tonearm cable together form an electrical resonance that generates Radio Frequency Interference which is injected directly into the phono section and can cause distortion. The loading resistor detunes the electrical resonance, thus eliminating the RFI.
However if your preamp was designed by someone that knows about this phenomena, you'll find that the highest load impedance (47,000 Ohms) is the right one, with little or no effect at lower resistances other than the tracking ability is reduced.