I've tried both, a dedicated fan less PC and a normal PC. I could not hear any difference. Even if a music player is the only running app, Windows still runs a plethora of background tasks. I would not worry about that. Besides, any 'dedicated' streamer or player that runs on an operating system does too.
I used a USB to USB galvanic isolator and that made a huge difference. It takes out literally all the PC noise. There also are USB to optical (toslink) converters on the market, 192kHz / 24 bit ... more than enough for normal music playback.