But there are choices that affect the sound. If you need a 1 watt 1K resistor you can use any type on a cathode, but different types have different sounds. The cathode bypass cap might need to be 100 uF. Different 100 uF caps sound very different. All will satisfy engineering standards, but parts and layout choices have profound effect on the final sound.
When you run zero feedback, the circuit has no ability to reject things like this. So everything makes a difference. However, for something like an electrolytic bypass, I think you'll find that as long as the part is good quality, the big differences you hear will be more about the part forming up over time: they will arrive at the same place sooner or later.
For reasons that are not clear, various brands of metallized polypropylene capacitors sound quite different from each other, and there is little correlation with DA and DF parameters. Based on measurements, they should all sound the same.
Sometimes you have to do your own measurements because the specs of the manufacturer don't always tell the whole story. If you use a precision differential amplifier to drive the caps in question, you can measure how they behave and differ from one another while in circuit.