Input impedance is like a pressure the other component has to work against. The other component can be a cartridge going into a phono stage, phono stage going into a preamp, preamp going into a amp, or amp going into a speaker. It is all the same concept.
If the receiving device has a low impedance, low resistance, then the component driving it will need to be able to deliver a lot of volume, aka amperage, aka current flow, to drive it. Or else it will "run out of gas" and you will hear this as a loss of musical drive and energy, and often times worse bass response. If the driving component is weak in terms of power supply then impedance matching becomes ever more important.
There is also another way in which impedance matching matters, and it works in a way that really complicates matters and confuses a lot of people. Send a current down a wire and it goes and goes pretty much the same the whole way. This we glorify and call transmission line theory. It goes and goes just fine, until it encounters a disturbance, usually a connection or the next component. At this point if the impedance changes a lot this sends a sort of shock wave back along the wire all the way to the source. Where if it again encounters another impedance change the shock wave bounces back again. This all happens very fast and produces a sound we call ringing.
Ringing happens most notably in MC phono cartridges where the high impedance is great. Because the phono cartridge cannot put out much current. But too high and the ringing accentuates the top end. So we "load it down" by lowering impedance until we get the sound we like.
Unfortunately something we cannot do with amps and preamps, thus these rules of thumb they keep coming up with, because it is a whole lot easier to tell someone a rule they don't understand than to explain it like I just did so they do.