Exhibit A
Excerpt from article in Stereophile Magazine July 2005 by Editor John Atkinson on the dodgy subject of blind tests and the validity of same.
“In the summer of 1978 I took part in a blind listening test organized by Martin Colloms, in which the panel tried to distinguish by ear between two solid-state power amplifiers—a Quad 405 and a Naim NAP250—and a tube amp, a Michaelson & Austin TVA-1. The results of the test were inconclusive, the listeners apparently not being able to distinguish between the amplifiers (see HFN, November 1978). Having been involved in the tests, having seen how carefully Martin had organized them, and having experienced nothing that conflicted with my beliefs, I concluded that the null results proved that the amplifiers didn’t sound different from one another. I bought a Quad 405.
However, over time I began to realize that even though the sound of my system with the Quad was the same as it ever had been, the magic was gone. Listening to records began to play a smaller role in my life—until I replaced the 405 with an M&A tube amplifier two years later.
The lesson was duly learned. Whether or not they can be told apart under blind conditions, amplifiers can have a major effect on a system’s sound quality. And more important, normal listening had revealed what the blind test had missed. I told this anecdote at the debate to make two specific points. First, it demonstrates that my following the then-as-now "objectivist" mantra—that audiophiles should buy the cheapest amplifier that offers the power and features they need—had let me down. Second, it pits against one another two core beliefs of the believers in "scientific" testing: 1) that a blind test, merely by being blind, reveals the reality of audible amplifier differences; and 2) that sighted listening is dominated by nonaudio factors, the so-called "Placebo Effect."
To explain my quarter-century-old Damascene experience, you have to accept that either the blind test was flawed—in which case all the reports that cited that 1978 test as "proving" the amplifiers sounded the same were wrong—or that the nonaudio factors were irrelevant, in which case the criticisms of sighted listening based on that factor must be wrong.”
Excerpt from article in Stereophile Magazine July 2005 by Editor John Atkinson on the dodgy subject of blind tests and the validity of same.
“In the summer of 1978 I took part in a blind listening test organized by Martin Colloms, in which the panel tried to distinguish by ear between two solid-state power amplifiers—a Quad 405 and a Naim NAP250—and a tube amp, a Michaelson & Austin TVA-1. The results of the test were inconclusive, the listeners apparently not being able to distinguish between the amplifiers (see HFN, November 1978). Having been involved in the tests, having seen how carefully Martin had organized them, and having experienced nothing that conflicted with my beliefs, I concluded that the null results proved that the amplifiers didn’t sound different from one another. I bought a Quad 405.
However, over time I began to realize that even though the sound of my system with the Quad was the same as it ever had been, the magic was gone. Listening to records began to play a smaller role in my life—until I replaced the 405 with an M&A tube amplifier two years later.
The lesson was duly learned. Whether or not they can be told apart under blind conditions, amplifiers can have a major effect on a system’s sound quality. And more important, normal listening had revealed what the blind test had missed. I told this anecdote at the debate to make two specific points. First, it demonstrates that my following the then-as-now "objectivist" mantra—that audiophiles should buy the cheapest amplifier that offers the power and features they need—had let me down. Second, it pits against one another two core beliefs of the believers in "scientific" testing: 1) that a blind test, merely by being blind, reveals the reality of audible amplifier differences; and 2) that sighted listening is dominated by nonaudio factors, the so-called "Placebo Effect."
To explain my quarter-century-old Damascene experience, you have to accept that either the blind test was flawed—in which case all the reports that cited that 1978 test as "proving" the amplifiers sounded the same were wrong—or that the nonaudio factors were irrelevant, in which case the criticisms of sighted listening based on that factor must be wrong.”