I too had the P5x, (not the e) for me as my primary phono pre for several years and kept it around through my upgrades. I was very neutral sounding to my ears.
As I upgraded TT, arm, cart, AND my ability to set it all up, so-so recordings that at one time sounded bright would become much more listenable. The whole process surprised me some. Not claiming any miracles though.
In my experience if equipment is neutral what you get out depends a lot on what goes in.
One caveat is that I only used ZYX cartdriges with the Ayre. And I ran it balanced which was clearly better IMO.