@prof, excellent and very thoughtful answer to my post. I agree wholeheartedly that if something measures differently (and I referred to REW, and not just S/N, distortion figures, etc., because that theoretically should give you lots of data points that show a difference when something is changed in your system, since something should change in some of that data if it sounds different). If you think that there is a difference, despite no objective support for it, A/B testing is the perfect way to eliminate confirmation bias. Can a difference in depth of soundstage, or positioning of instruments within a soundstage be objectively measured? I don't know the answer to that. What role does confirmation bias play in me believing that I hear a difference? I don't know the answer to that either, although I like to think that I am being objective. I replaced an Audience Power Chord that I had used for a long time on my DAC with a $$$ Shunyata one (same length, but slightly lower AWG after liking what they did for my Preamp (over stock) and speakers. I can honestly say that I hear no real difference . . . maybe an ever so slight improvement, but that could definitely be representative of a struggle to hear a difference because I expect/want a difference -- the paradigm of confirmation bias.
But as you accurately point out, whatever the answers are, A/B testing should yield a consistent result that eliminates confirmation bias. That is why I am surprised that A/B testing is not more widely accepted here on AudioGon. A very interesting thread as putting many manufacturers' claims to objective tests has always been an interest of mine. Thanks for the thoughtful response.
@crymeanaudioriver I personally would love to participate in A/B testing for anything, but especially the seemingly wild claims of things like the Shun Mook, Stillpoints that you scatter around, risers that elevate cables off the floor, that weird brass salad bowl that "snaps the soundstage into focus," etc.. . every manufacturer has some sales patter regarding how their tweak real works, but no one really cites either objective measurements or the results of A/B testing. That would be a lot of fun! Maybe some tweaks manufacturers would be willing to do some A/B testing at Axpona?