More expensive/higher standard power supplies just mean they are more efficient in terms of power consumption overall. They produce less heat, may last longer, and sometimes improve overall system performance (in the case that you a have a powerful gaming PC, I’ve built these and others etc. severs before...my day job is IT).
In terms of the PSU itself, I don’t think that will impact sound quality. Because it’s a computer, not an audio component.
All good DACs will pre-filter the USB input from any noise over USB, so that’s not a concern.
In regards to storage options for music, I would suggest a fast and reliable SATA III SSD of your choice (capacity) or a PCI-e /nVME solid-state drive. If the motherboard on your computer is not already compatible, you can buy a PCI-e card and add that functionality by using one PCI-e lane on the motherboard. Hard Drives are noisy! bad for music.
The quality of electricity (especially of AC) before it reaches an audio component or anywhere within that chain is important.
Check out my profile to see what I’m using. And feel free to send me a message.