Miklorsmith: I agree (and have detailed before) that there can be problems with formal testing methodologies as applied to subjective auditioning. I do think some kinds of testing can introduce a "confusion factor" that may actually serve to artificially raise the floor for perceivability of low-level differences. And I think it's to a large extent possible to ameliorate biasing effects due to external factors without resorting to blind tests, though it can take repetition over time and a certain self-questioning mindset (that I'm learning a lot of audiophiles seem to lack). As for how test conditions might significantly differ from normal use conditions, this can be good or bad -- I don't listen to music for enjoyment by performing rapid A/B comparisons, but doing them can really help nail down (or dismiss) some elusive observations concerning gear.
But, when faced with a product or claim that appears to carry all the hallmarks of snakeoil, and audiophiles buying into it using the most casual and fallible auditioning methods, I don't think it's inappropriate to call for some demonstrable degree of rigor to be brought to bear. I also think that experiences like the one you relate above are valuable for putting things in proper perspective every once in a while.
But, when faced with a product or claim that appears to carry all the hallmarks of snakeoil, and audiophiles buying into it using the most casual and fallible auditioning methods, I don't think it's inappropriate to call for some demonstrable degree of rigor to be brought to bear. I also think that experiences like the one you relate above are valuable for putting things in proper perspective every once in a while.