Why is Double Blind Testing Controversial?

Well said Bomarc. I'll go a little further along that road. Most people who argue in favor of DBT's in these threads imply that audiophiles don't want to discover that they can't tell one thing from another. I think audiophiles believe that DBT's are not good because that's what they've been told by the buff mag writers (aka "reviewers," but really just story tellers), who don't want to be exposed. You see, Sean, in a DBT, Mikey can't tell those preamps apart either.

There are reviewers with real scientific credentials and experience who advocate and use DBT's, but not many.

They don't have to be short. You can listen to whole pieces of music. And you can start by familiarizing yourself with the components under test until you are sure of what it is that makes them sound different, then try to tell them apart blind, not trying to prove there is no difference, but to confirm your hypothesis that they are different. In home, one obvious difficulty is exact matching of spls.

I have done blind testing myself, on many occasions, as I have mentioned before, in an earlier thread on this subject. When I worked in a high-end audio shop, when there were slow times, when nobody was in the store. Myself and the other employees would do it for fun. It was like a contest. We would set up equipment when 1 person was out of the room, blindfold him, and bring him in to listen. Then we would see how many components in the system he could identify. Everyone was surprisingly good at this, but nobody was 100%. Of course, we were going for multiple components at the same time. With only 1 component in a known system, we all could indentify it virtually every time. Now, we were very familiar with the sound of all of the gear we tested this way, so that made it easier. But, it wasn't that tough to do. It was in a fairly relaxed environment with no real pressure. Just fun.

So, I am not afraid of what would be the result of these tests, in my case. I cannot say I could be 100% in any test, but I am sure that I can be accurate enough to satisfy any tester that I am not guessing. Even with unknown equipment, I can identify differences accurately, in a short time span. I think many people can. I also think that some people cannot. This would not be the lack of difference in equipment, but a difference in people. It is not scientific to label the results as "no difference" in equipment, when it is an inability on the part of the listener to percieve the difference. Many people think a car radio sounds like a good audio system. Clearly they are not listening to the entire presentation, but just the superficial aspects of the sound. They are not aware of how to listen for "differences" in the sounds of items. They are simply "superficial listeners". This type of listener cannot be relied upon to discern differences in equipment. They listen as "background music". A person who knows what to listen for, will easily tell the differences in components.

And again, I will state that I think that this whole issue is a "red herring", that is brought up by those who have convinced themselves that there is no difference in equipment, or cables, or whatever, in order to satisfy their own minds that they do not have to spend money on such things, and still have the best. In my opinion, the desire to save money is stronger, even in the most ardent audiophile, than the desire to spend a lot of money fruitlessly. I would say that I, and everyone else here, would rather spend less, to get the same performance, if we could do it. And there are ways to do it. The most expensive item is not always the best sounding item. But sometimes it is. And making claims that "in a real scientific environment we couldn't tell the difference" is simply a diversion.

In virtually every case where this is mentioned, it is in a context of "wasting money on snake oil". This is the crux of the matter. It is a matter of justifying expenditures.

So now, we have a whole different scenario. Now we have a subject brought up, which is the real center of the matter, which is,"Is it worth it to me to spend alot of money to get a certain level of performance increase, when I am not sure of the outcome?"

There's such a plethora of cables and components, and claims of grandeur, that some people cannot cope with it, and punt. Instead they divert their attentions to claiming that there is no difference between these things, and stick to it. They use a "scientific" argument that they know nobody is going to use, to back-up their idea.

In some cases, they will be right, and there will be no noticeable differences in some items. This only adds to the confusion, because it lends credence to the extrapolation that there is no difference in anything.

If these folks want to believe that, then that is their prerogative, and they are entitled to believe that. But to tell the rest of us that we are "deluded" by our unquenchable desires to spend money, that we would manufacture these differences in our heads, just so we can spend more money, is not passing the "smell" test.

I'll use myself as an example. I "claim" that I can hear differences. But I don't want to spend any more than I have to, in order to get the sound quality I want. I can honestly say that I may hear very small, or even no, sonic difference between certain items that have significantly different prices. When I arrive at a situation like that, I call the lower priced item a "bargain", and I buy that one. Or I may say,"I like that one better, but the small difference is not worth the extra money to me." Isn't that a more "measured" approach than what we are seeing here? That is what everyone else does. When my girlfriend goes shopping for a dress, and she sees one she likes for alot less than a similar one, she remarks on what a "good deal" she got. She doesn't come home proclaiming that "there is no difference between dresses" so she bought the cheapest one in the store. After all, they all perform the same function of a covering, right?

All of this is much ado about nothing. People will buy what they want. Then they will justify it to themselves, or others. That is life.

Twl, thanks for taking the time to clearly articulate what is obvious to many of us, some products are superior to others and the decision is usually, whether the improvement is worth the cost. Also, thank God, there ARE superior products that cost less than the competion. Then there is also the issue of synergy and the proper matching of components. Judging from your system profile, it's obvious you are doing your homework. God bless.

TRUTH is controversial, to be sure - follow this link Audio Asylum post concerning hyperbole

Blind and double-blind is a way, as we all know, to attempt to remove the subjectivity from what is hoped to be an objective evaluation. It's great for testing some things in which there is a clear hierarchy of "poor" to "excellent." Eyes, ears, and taste buds, however, don't conform to objectivity. Try a blind wine-tasting to see how many diverse opinions are being swished around the tongue and spat out into the bucket.

I tried a blind (but not deaf!) test last night, as it so happens, of some Vampire vs. Nordost vs. van den Hul interconnects.

It's REALLY hard to change cables with one's eyes shut! I knocked my turntable onto the floor, stepped onto the open try of my CD player, connected the phono stage to the tape deck, smashed my 180gm Dylan LP, and shoved my index finger through the cone of my Cabasse.

After all that, all the cables sounded like s**t. In truth, the van den Hul bested them all, followed (at some distance) by the other two. The $15 Vampire sounded no different than the $150 Nordost.

Blind testing is fun, as Twl notes, but very difficult to implement in real life.

What an interesting thread. I've othen wondered if there would be value in SB/DB testing and why it isn't done more often. It seems to me that the reason for such tests is twofold, firstly to identify on an objective basis specific differences between components - eg transparency, ambiance, soundstage, resolution, focus, brightness, transient attack, clarity, etc; and secondly which component gives greater musicality over a period of time eg which gives the best pleasure, which is nearest to the original recording, which is nearest to live sound. The problem is that everyone hears sound differently. That's why some like the sound of the Festival Hall, and can't bear the sound of the same orchestra playing the same piece under the same conductor, in the Albert Hall (I live in the UK). I suggest the success or failiure of SB/DB testing would depend on its its ability to allow the listener to consistently pick out the component which gave them the greatest pleasure, or produced the specific type of ambiance, transparency, etc. that they wanted.

Most people can tell the difference between listening to components placed on a bog standard shelf in the living room, or on specifically designed audio stands such as the Sistrum, or Townshend: whether they prefer one over the others is dependent on how they hear sound. It's the same with cable burn-in: I defy anyone with normal hearing not to agree there is a difference between a fully burnt-in cable and a virgen cable. But that difference may be totally unimportant to them.

So maybe BS/DB testing at an individual level would have value, if one had the time and the money to spend on it. Otherwise, perhaps the knowledge - gained by quick A/B comparision, reviews, and most importantly people's opinions on forms such as this one - that component A has slightly more of what we want in terms of amibiance, transparency, etc; is about as far as we can reasonably expect to go, and very probably good enough for all but the most "golden-eared" of us.