Is ground noise really normal? If so, why?

Further to what @pragmasi said, ensure there are no routers, cable modems, or Wi-Fi units nearby. Move them with the volume up and note the change in hum levels.

If you truly own an Ayre P5Xe, you are missing out.  You have a true balanced phono stage, and you should be connecting your cartridge to the input in balanced mode.  To do that optimally, you want a true balanced phono cable, which is a cable that uses identical conductors for each half of the balanced signal; one conductor being connected to each "end" of the cartridge output.  Cartridges are inherently balanced devices (at least 98% of them).  The pin on your cartridge that is labeled "GD" (for ground) can actually be used to derive the negative phase of a balanced signal.  The pin labeled "hot" or "R" or "L" will then put out only the positive phase of a balanced signal.

So, basically, to convert to balanced mode all you need is a balanced phono cable with XLR termination.  You would connect the XLR to the XLR input on your Ayre phono.  This will eliminate any noise due to poor grounding, totally.  As others have correctly noted, this will not eliminate groove noise, etc, etc.  But you could say good-bye to 60Hz or 120Hz hum due to poor grounding. If that is the real source of your problem.

Now, if you tell me that your cartridge is already connected to your Ayre in true balanced mode, then I would say you do not have noise due to poor grounding.

'Ground noise' sounds like a bad ground connection to me.
If you have a balanced preamp then the ground of the arm should also be the shields of the tone arm interconnect cable and tying to pin 1 of the XLR connections on your preamp. The '+' and '-' outputs of the cartridge then connection to pins 2 and 3 of the XLR input connectors.

Ralph, First we need to determine if the OP is truly experiencing noise due to bad grounding.  I'm not sure.  But in balanced mode, I've never had any audible grounding issue, no matter how sloppy I might be with grounding the tonearm or turntable or etc.

As others have alluded to the OP should clarify what he means by "ground noise." Hum? Hiss? Something else? Is it present only when the stylus is in the groove of a rotating record, or is it also present when the stylus is lifted off of the record? Or is it present even when the platter is not rotating and the turntable is turned off?

Also, what cartridge is being used, and what gain and input impedance is the phono stage set to? For example, if the cartridge is a moving magnet type and the input impedance is set to 100 or 1000 ohms (the two choices that are provided in addition to the 47K setting that should be used for moving magnet cartridges) it would certainly result in poor signal-to-noise ratio.

But to provide a preliminary answer to the initial question, I would not expect an Ayre phono stage to have objectionable noise levels unless there is something is wrong in the setup or the unit is defective.

Regards,
-- Al