In defense of ABX testing

"50 to 100 samples you say? Why not make it 100 to 200? Over how many years do you expect your ABX listening test experiment to take?

Just curious have you ever A/B compared 2 or 3 cables to one another? More than 3 or 4 at a time? Could you hear audible differences between the cables?"

Yes, I have. I did an experiment a few years ago and compared AQ Cheetah IC's to a pair of AQ Panther IC's. Both cables are identical except for the conductors themselves. One silver, one copper. The goal was to see of a difference could be heard between the 2 metals, and nothing else. It wasn't about what one sounded better, just if there was a difference. There was 4 of us took the test and we listened to 100 samples of a 10 second audio clip that took around 30-40 minutes for each of us.

"50 to 100 samples.... Do you believe there are people in the world that can tell which key of a piano is struck on a tuned grand piano in a blind test? Do you think their brain learned the sound of each key in the span of a week or so, or even a few months or so? How about in a year? "

Actually yes, and I can prove it. My brother has something called perfect pitch. He can tell with 100% accuracy what any note or cord played on any instrument is, and if its in tune or not. I don't have it myself, but if you have ever played an instrument, you can develop something called relative pitch. Its not as good as perfect pitch, but its a skill that can be learned. For me, I needed to develop the skill somewhat when I played drums in school. If you have ever seen kettle drums or tympani, they have to be tuned to a certain note when you play them. That is what the food pedal is for, it sets tension on the drum head. Anyway, you have to be able to set the drums to different notes while the band is playing. To do this, you tap it very lightly (because the band is playing), and hopefully tune it to the right note before you need to play it. Its not an easy thing to do, but its a skill that can be learned.

This to me is very indicative of the people in power or the ones that are the "experts" wanting to stay that way. Remember the attitude in the sixties and early seventies regarding wines and how the "experts" continuously stated that French wines were the best and everyone else's was not very good? It wasn't until the "judgement in Paris" happened that the world realized that opinions were changed dramatically when blind testing occurred. There I absolutely no scientifically logical explanation why blind testing isn't the best comparison method.

Of course it has to be an apples to apples comparison. This to me means price point testing. Just like cars. Pick a price point, get the equipment that falls within that price range and go at it. But, tube lovers will pick tube equipment most of the time based on knowing what they are hearing ahead of time. Same is true for solid state lovers. But, blind testing? within price points? Lets see what the experts say then. But, the "experts" don't what to do that because the would show people that many of them (absolutely not all of them) are frauds.

if test are not done scientifically and are not based on "opinions" they really aren't real to me. How does one measure whether the equipment accurately demonstrated the sound stage depth? dimensionality? etc. I hear many opinions of the reviewers, but based on what? What criteria? are you going by memory in your opinions and comparisons? or did you listen intently and then switch out that amp with another (without changing anything else) and listen again?

I have read of some reviews that do exactly that. And the equipment they are reviewing is compared to similar equipment within the price point. That is alright for me. But, I still prefer an A/B comparison test that is blind to really identify the sonic differences in an unbiased way.

enjoy

"And the findings, results, of the listening test?

Just curious what is behind your thinking of needing so many samples for your listening test?"

The real issue here is that you don't care about any tests that were done. You've got your emotions tied up in all this and just want to win the argument, and be right.

You ask me why I needed so many samples for my tests. Not only did I already give the the answer in a prior thread, you quoted it in your last post! Here it is. Maybe you'll remember it this time.

"So, for example, if you were trying to test to see if a difference can be heard between a silver cable and a copper cable, all other things being equal, maybe have them listen to 50 or 100 samples. Maybe they can get lucky and guess correctly for 5 or 10, but 100 is highly unlikely. "

I thought I was pretty clear, but I'll try to explain it again. With 5 or 10 samples in a simple yes or no test, I thought it wouldn't be out of the question to get an inaccurate score due to error's or guessing. A small sampling really leaves no margin for error. I mean if I flip a coin 10 times, what are the chances of you getting 5 heads and 5 tails and average 50% like the test should? There's a very high probability you won't average 50% with such a small sample. Flip the coin 100 times and you will get much closer to the statistically accurate 50% that you should be getting. Now, just 1 last time to be extremely clear, If I flipped a coin 10x, you have a much greater chance of guessing something other than 50% than if I was to flip it 100x. That's why I needed so many samples. Didn't you have to take statistics in college?

And as to the results of the test's, its not relative to this discussion. You only want me to list the results so you can comb through them to find the slightest detail just so you can claim the whole thing is null and void, so you get to be right. I won't play that game. You're just going to have to continue playing with yourself like you've been doing.

After my last post something tells me I won't get anywhere, but here go.

"if test are not done scientifically and are not based on "opinions" they really aren't real to me. How does one measure whether the equipment accurately demonstrated the sound stage depth? dimensionality? etc. I hear many opinions of the reviewers, but based on what? What criteria? are you going by memory in your opinions and comparisons? or did you listen intently and then switch out that amp with another (without changing anything else) and listen again?

I have read of some reviews that do exactly that. And the equipment they are reviewing is compared to similar equipment within the price point. That is alright for me. But, I still prefer an A/B comparison test that is blind to really identify the sonic differences in an unbiased way.

In the first paragraph, you're talking about subjective qualities that the reviewers are discussing. We all know that those qualities mentioned can't be measured, so what would you have the reviewer do? We're supposed to be adults here. When I read a review its not too difficult to pick out the things that are purely subjective in nature. Yes, they are listening to the component and writing their subjective opinion as to what they heard. Here's the 1 detail that many people miss. Most of the people that read the reviews, the magazines customers, know this is how they do it, and its not a perfect process, but they still want the review anyway. Any why not? Why do you think they bought the magazine to begin with?

This caught my eye in particular.

"But, I still prefer an A/B comparison test that is blind to really identify the sonic differences in an unbiased way.
"

You say that you prefer this type of blind testing like there are some reviewers that are doing it. I've never seen any reviewers do this. Where are you finding them? I'm more than willing to give them a chance. If they can show me some testing that helps make a better decision, I'm all for it.