I’m not too sure what causes it but I don’t think it can be the proximity of the various parts.
I’ve used a tiny Lenovo PC hardly bigger than a paperback that has no interference whatsoever through headphones.
Same for many tablets and laptops.
Perhaps it’s a question of inadequate shielding? Or even a particular motherboard issue?
Although my first PC had a specialist soundcard I don’t think it sounded any better than the later ones with integrated sound chips.
I also prefer to use the minimum software (W7 driver) and stick to WMP and VLC - no enhancements / EQ.
The big issue seems to be the speakers and headphones you wish to use. It's far from easy to find a pair with a reasonably flat frequency response that you could trust for monitoring or comparison purposes.