atdavidWhile ideally the test would be double blind, that is not always a viable thing to do.
Quite so! Conducting a controlled test with the goal that it will be scientifically valid is a tricky business. It’s time-consuming, tedious and cumbersome, so not likely to be fun for most audiophiles. I’m sure that’s one reason such tests are uncommon in our community.
There are two biases, subject bias and observation bias. Single blind removes subject bias, which is usually the dominant bias. While double blind is the "gold standard", single bias is still used as it eliminate subject bias, and provides significantly more statistically relevant results. To suggest this is no more valuable than sighted tests shows a gross ignorance w.r.t. this type of testing.
Sorry, but you don’t know what you’re talking about. You can’t allow potential bias to creep into a test, and then excuse the bias by proclaiming that isn’t the "dominant" bias, whatever that is. The test you describe is a sighted test, and subject to whatever flaws such tests might include.
That’s especially so in this instance, where you’ve acknowledged that the tester will have to "lie" to the subject about whether he might be hearing the A or B component. The tester’s bias - or even his personality traits - can poison the test. For example, the tester may reveal a "tell" (as a poker player would call it) that could be even unconsciously detected by the test subject, thereby influencing his responses. And that’s why I said that your partially-sighted test has no advantage over a fully sighted test - bias can affect results either way.
If you want to conduct a scientific test, you have to control for as many variables as possible. This is science 101; there’s no disputing this basic stuff. I think it’s rather odd that some of those who clamor most loudly for scientific listening tests have such poor understanding of what’s involved. Perhaps that’s why they are so confounded that controlled tests are so rare: They think such testing is simple and straightforward, while nothing could be further from the truth.
