Simplest Linear PSU?

In my understanding, source components for obvious reasons require the most silent power supplies.

Absolutely. I go nuts getting really quiet power for moving coil RIAA phono stages. Think about the impact of even a small amount of noise on something that may be amplified 20,000X by the time it reaches the speaker. How do i get that number? In farmer’s math, my MC stage has a gain of 66dB, 2000X, my power amp about 20X, preamp about 3X, but of course the volume is mostly turned down significantly, lowering the effective gain. BTW when i say "my" they are both what i use but also designed by me - so i know them well. My daily drivers are all prototypes of some form, some from long ago.

Given the above 1/10,000th of a volt of input noise becomes two volts of output noise. Which is normal listening level! It would be all noise! A signal-to-noise ration of....... 1. :-) Puts things in perspective eh?

DACs are actually much more forgiving, since the voltage levels on a DAC are nominally much higher.  But still you are right - whatever goes into the beginning of the signal chain gets amplified.

Regulation done right typically reduces noise. Done wrong or badly or with defective parts, I suppose it could add noise. Even for a home built LPS, there are excellent prefabricated regulators available as add on devices. Most important is to plan ahead for voltage and current needs and to account in advance for the headroom that any regulator needs.

Another Teddy Pardo. Made a noticeable difference over the stock SMPS.

@OP

Being smaller, generally, SMD components are actually more "discreet":))

Both SMD and through hole devices can be "discrete".