Heh, sometimes you don’t realize something sounds harsh/sterile until you hear it without that harsh/sterile character. That was the case in my situation as I used to use all Hubbel hospital outlets and male plug connectors (all brass). I was working with different gold-plated connectors on my system upstairs and just for the heck of it, I popped in a Wattgate brass connector. The result was fast and clear, but it was definitely more sterile/cold sounding (less emotionally engaging and less organic). I suddenly thought "hey, this is what my system downstairs sounds like!!" Then I removed all brass elements in my A/C line completely!
As far as the outlet question, an upgraded outlet will definitely improve/affect the sound, but not as much as a power cable. Generally, elements in the A/C lines that are closer to the equipment power supply will have more of an influence on the sound. That’s not to say that an outlet won’t make any difference, it will just be more subtle.
I generally recommend that people start at the equipment power supply itself, beginning with upgrading the fuse and then work outward from there. The fuse will have a significant impact to the sound quality, but the fuse and power cable work hand-in-hand so both are very important.
In your case, if you decide to upgrade the DH Labs cable to rhodium connectors, I would upgrade the fuse in the amp to a rhodium plated Furutech at the same time. Then you can burn in both the fuse and power cord connectors together. You can upgrade the fuse in the headphone amp also, If you wish. I could not see if the PS Audio DAC uses a fuse, so that is unclear.
At this point, it really depends on if you connect all your components straight to the wall or if you use a power strit in between. The typical power strips are horrible and will further degrade any improvement you have. They can have thin/cheap metal conductors internally and small gauge wire (such as 18awg). It’s always best to connect straight to the wall (especially for the amp). If you must use a power strip, look into a very good one. The Emotiva CMX-6 is very good for the money and has a nice small filter/conditioner built in. You can also upgrade the power cord for it to something better if you like. However, be aware that the internal wiring on the Emotiva is 14awg. If you want something better, it will cost you $$$$.