How much difference does a phono preamp really make?

It can make a huge difference.

First fundamental step is make sure phono input specs match cart output specs ie the two are compatible. Then you are baselined optimally to assess any other differences from device to device, if it still matters enough to you.

Of course its a system so everything else downstream from pre-amp to speakers to setup in the room matters as well. The trick is to figure out the weak link and focus on that as needed. Don’t get too hung up in one place.

If that sounds hard, then consider one the many integrated solutions available today and there are many very good ones capable of competing with many high quality separates. The engineers ie the technical experts have integrated the things that matter for you there. That can help take a lot of the guessing out of the game. You still need to pick a cartridge that is compatible and in many but not all cases speakers as well.

@ audionoobie OP

I can hear a difference between the two carts. Not so much with the two phono pre's
It seems like my carts may not be good enough for me to notice a difference between my entry phono stages.

OK, that changes the answer for me. Both preamps are about equal sonically so you won't hear a difference in them. But the carts are doing their job. You can hear a difference.  Though your carts are not really high end, they will respond much better with a better phono preamp. Remember the hierarchy I posted. The cartridge was the least important. 
Atmasphere noted as much telling how a lesser cartridge sounded great on a better arm & TT. The phono pre will also give better and is where I'd spend my $$$ now. 

I have experienced many phono stages, and in general they are extremely important in the system as a whole. They are far up the signal path and dealing with a minuet signal, they must be very quiet and high quality to do it well. However, also the basic rule of thumb applies to Phonostage… “you need to double the cost of a component to get a solid very noticeable improvement” (its just a rule of thumb… but typically after you get into components over a couple grand it works pretty well).

For me, there was a rather massive improvement in going from a Schiit Mani to a Rega Aria Mk3.  

@audionoobie - I had budget phono stages for many years and the improvement from one to another was marginal - even when I tweaked power supplies - the improvements were incremental.

I then decided to invest in the analogue side of my system and purchased an LFD phono stage, but it was not a good match to my system or to my cartridge.

Fortunately I was able to switch at no cost to a Simaudio Moon LP5.3 RS and boy - what an improvement.

So much so that when I had my Denon 103 re-tipped by Soundsmith the improvements were immediately apparent.

Since then I have upgraded power cables and interconnects several times and the MOON has kept pace with all those changes by conveying far more details than I thought were possible.

So

• your choice of phono stage choice is critical
• matching the phono stage to your system is recommended
• matching the phono too your cartridge is is recommended
• and matching your cables to your components is recommended
• i.e. if you want to get the very best out of your analogue components.

BUT - I consider the most important cable, is from the cartridge to the phono stage - I switched to a one piece harness many years ago - i.e. basically 4 single wires from the cartridge to the phono stage - within no RCA’s or joins.in the wires

Cardas makes a very nice one piece harness.

Once you get the cables sorted, the difference between phono stages should be more noticeable

Just another opinion - Steve