Upgrade to a better server. Ideas?

But then who would host roon core?

Im not sure if a different server will make a significant difference in sound quality but if you have a "larger" music collection you might want to give Andrew Gillis at Small Green Computer and look at one the i5, i7, or i9 servers.  Also consider  watching a few of Andrew's video or better yet email / call him about cabling and the use of fiber.  The X1 is a good piece of equipment.  I would see how he suggests leveraging its capabilities.  

I'm not associated with SGC just very happy with their customer support and equipment. 

Check the Innuos line of servers, since you use Roon:

https://innuos.com/zenith/

The zenith has streamer capabilities which I don’t need.

I am only focusing on the server.

If the server has the streamer it may impact Signal delivery and I already have a streamer as part of the lumin x1 which seems fine

Bryston CD player.