In defense of ABX testing

I agree that only a limited number of switches are needed, if the test conditions are good. What are good test conditions?: A treated room with good acoustics, high quality electronics, well-recorded music, the ability to do rapid switching (having a second person to manipulate the hardware helps), and familiarity with the musical selections. That's all you need to eliminate subjectivity and get to the truth.

"01-29-15: Psag
I agree that only a limited number of switches are needed, if the test conditions are good. What are good test conditions?: A treated room with good acoustics, high quality electronics, well-recorded music, the ability to do rapid switching (having a second person to manipulate the hardware helps), and familiarity with the musical selections. That's all you need to eliminate subjectivity and get to the truth."

What I was referring to was a very simple test. You have 2 cables in the system, 1 is copper, the other silver. The goal was to see if you could pick out the silver or copper, and that's it. Nothing subjective like what cable sounds better. That's just personal preference. So after you hear a 10 second clip of music, you say copper or silver. With such a small sample, you can't really weed out things that may produce bad results. For example, lets say that there really was no difference that a test subject could hear between the 2 cables. That would mean, both cables would sound identical. But we won't know that until after the test. That would also mean, every answer given would only be right by pure chance. So for this test, since there are only 2 answers, and going by the assumption that there is no difference, over time, the answers would have to conform to a 50/50 split. If we only got 10 samples under this scenario, there's a really good chance you wouldn't get a 50/50 split with just 10 tries. With 100 tries, you get much closer. An easy way to visualize, or even try this concept to see for your self, would be to filp a coin. Flip it 10x, and even though you should get 5 heads and 5 tails, with so few tries, you can easily get different results. The only way to reduce this type of error is to take a larger sample. Flip a coin 100 times, and you'll get much closer to the 50/50 split that you would expect to get from just pure chance.

Zd542, I think statistics is somewhat of a pseudoscience, so I can't completely address what you are saying, other than to offer my own experience. When there is a significant difference between two listening test scenarios, that difference is quickly evident, and it will be evident each and every time. Were that not the case, the difference would not be significant. Obviously I'm not talking about the textbook definition of 'significant difference'. There's not much point in making changes to an audio system if the differences approach the statistical limit of significance.

Psag,

Sorry, but I don't buy your last post. It makes absolutely no sense. You mean to tell me that the concept of probability, a simple coin toss, can't be addressed buy you because you feel its pseudoscience? You don't have to be an actuary to understand the concept, they teach it in grade school. And then to go on with all the subjective issues when I clearly stated that the whole purpose of the test was to was to not go there.

Given the above, the only thing that makes your response understandable, is if there is some type of deceit involved. I guess that maybe you don't like me from another thread, or you have a technical background that your ego needs to support, or something. Whatever it is, I don't believe you don't understand something so simple. I believe that you won't understand something so simple.

Sorry, but I just don't see it any other way.