A good quality breaker costs about $40 and replaces the on/off switch that carries its own cost. No, a great quality circuit breaker will not impact the sound any more than a quality power switch.
Another big area for improvement in amps are the binding posts. Want to hear an improvement on the order of what these high end fuses deliver? Ok, get rid of the binding posts! No kidding. I know most won't ever go for this for many reasons, but the improvement is substantial. Ok this is for out of the box designers and builders. Build tube amps/amps with the output transformer leads running out to a nicely designed Teflon post that is built to clamp the tranny leads to the speaker cable. The only role the Teflon or wooden post plays is to clamp! Certainly a professional designer could design such a part that is simple to use and rugged. I use Teflon bolts, nuts and washers to accomplish it. Very notiicable and like all fuses, connectors no matter the cost, all degrade sound. Yes I hard wire my power cord and ICs to my DIY tube amp for best sound:)
