Given the light discussion around ab(x) testing, as a scientist with phd training in human data collection and cognitive functioning, I object to the (x) part of the judgment task.
Identification should not be the goal. We listen to equipment to decide which we prefer... which is better. That should be the decision when doing (blind) ab testing.
If I were designing the test for audio equipment, I'd let people switch back and forth as many times as they like, taking as long as they like for each (level matched) sample. Eventually they just decide which they like best. Repeat that task over and over throughout the course of listening to music and I guarantee that people will begin to find their true preference and that that preference will be consistent.
It's very easy for the brain to develop and make decisions based on good/bad/preference and much harder to make the decision based on label identification. They are different processes in the brain and preference is the more basic, a primary system. Newborns know what they like and what they don't and will often show good a/b consistency. The brain doesn't even need to identify a stimulus to make this judgment. That comes later because it's less important. First, figure out if something's good or bad, then if you've got time, figure out what it is. This is the order of processing for all of us.
The eye doctor uses this ab-preference testing when they're figuring out your prescription. They flip between two possible magnification factors and you just tell them which is better. It works great for letting you find your own way to an optimal solution. They don't care about labels or giving you mystery options and having you say whether it corresponds to a or b because that's irrelevant. It would make the process less effective.
And finally, I'll just say that if you don't believe in blind testing, you're lying to yourself. There's decades of research demonstrating the impossibility of avoiding the biasing effects of pre-existing knowledge. People have used samples of scientists trained in this area of research and they are just as influenced as you or I. You are not immune.
That said, it's hard as hell to do blind testing and I never do. Fortunately placebo effects are very real and they help your preconceived decisions feel right even if they were wrong.
Identification should not be the goal. We listen to equipment to decide which we prefer... which is better. That should be the decision when doing (blind) ab testing.
If I were designing the test for audio equipment, I'd let people switch back and forth as many times as they like, taking as long as they like for each (level matched) sample. Eventually they just decide which they like best. Repeat that task over and over throughout the course of listening to music and I guarantee that people will begin to find their true preference and that that preference will be consistent.
It's very easy for the brain to develop and make decisions based on good/bad/preference and much harder to make the decision based on label identification. They are different processes in the brain and preference is the more basic, a primary system. Newborns know what they like and what they don't and will often show good a/b consistency. The brain doesn't even need to identify a stimulus to make this judgment. That comes later because it's less important. First, figure out if something's good or bad, then if you've got time, figure out what it is. This is the order of processing for all of us.
The eye doctor uses this ab-preference testing when they're figuring out your prescription. They flip between two possible magnification factors and you just tell them which is better. It works great for letting you find your own way to an optimal solution. They don't care about labels or giving you mystery options and having you say whether it corresponds to a or b because that's irrelevant. It would make the process less effective.
And finally, I'll just say that if you don't believe in blind testing, you're lying to yourself. There's decades of research demonstrating the impossibility of avoiding the biasing effects of pre-existing knowledge. People have used samples of scientists trained in this area of research and they are just as influenced as you or I. You are not immune.
That said, it's hard as hell to do blind testing and I never do. Fortunately placebo effects are very real and they help your preconceived decisions feel right even if they were wrong.