Out of Control

Gallaine, I did what? I don't even own a gauntlet. I have not waived math since high school, and have never waived it as a sceptre. Nothing so aristocratic for me. I like the expression "arts & sciences" though and I think it applies to audio equipment quite well. I won't trouble you with a dissertation on trees falling in the forest with nobody there to listen. I have no intention of talking about paradigms or the lack thereof. I will not even mention that a bad theory is better than none at all. There are many ways of getting at the truth. When it comes to understanding the physical phenomena that surround us, I doubt anything has rivalled science thus far. Magic thought certainly hasn't. I would merely suggest that audiophiles should show some method to their own madness and have some sort of basic procedure when evaluating equipment so that the fewest variables are introduced. Maybe this standard should only apply to professional reviewers, I don't know. It seems to me unfortunate that comments, good or bad, can be made about equipment heard on the fly or in less than good conditions (unknown room, unfamiliar music, changing the music used every time and the list could go on). By the way, I do not own, nor I have I ever owned, an oscilloscope, spectrum analyzer, calibrated microphone, SPL meter, fast Fourier analysis computer or other such piece of equipment or an ABX box for that matter. I would certainly appreciate it though if the manufacturer of the equipment I buy does and uses them, and interprets the results properly. If the manufacturer stops there we may not have the sound we want, we do appreciate that the component be evaluated by actual humans, listening to music and that suitable tweaking be done to have it perform at its best. I simply doubt the manufacturer could get to the tweaking stage without benefit of the hard data to start with.

If I might insinuate myself into this conversation between Pbb and Gregm, I would say that I don't object to double-blind testing. It certainly is a means to guard against the power of suggestion. However, I don't see that it much applies to listening evaluations of amps or any other audiophile equipment. Many of us are pretty sure of what we hear, so any double-blind test would be for the non-believers like Pbb rather than for us. However, I will gladly participate in any double-blind amp evaluation test that Pbb would like to set up. If he would just send the airline ticket to me so that I can be at the location of this test ... As far as the sound quality of power amps are concerned, two amps, both highly respected and of similar quality quite often sound very different, in other words easily recognizable from one another, using double-blind testing or not. I would agree that speakers show more sonic variation than amplifiers, but amplifiers are still easily discernable and identifiable from one another. I think that because amps "measure" much better than speakers, they are considered to be much more accurate reproducers of sound. And, if they are more accurate, they must sound the same, very similar, or at least so similar that few can tell the difference between two high quality amps. I think that the mistake here is that what is measured for either power amplifiers or speaker systems is not necessarily the final determinant of what the human ear can hear or all the aural information the human brain can process. Eighty to ninety percent of the measurements taken today for amplifiers (or speakers) they were measuring 50 years ago and while amplifiers have improved by orders of magnitude, they still don't sound live or anywhere near perfect, or even that much alike. So Pbb, let us know where the double-blind test is to be held.

Pbb: It was Bryston 7s & Accuphase 1000 (not 2000). My mistake for sounding general. I was just detailing the system & surroundings, as you had asked for in your 11/9 post.
Thanks for yr latest informative response. I agree with your experiences re, shows and dealers and, indeed, your commendable rigorous approach to judging equipment, etc.

Cheers!

Pbb, I would be interested in knowing what kind of evaluation you perform when choosing between components. My post wasn't meant as a defense for either side of the issue. But too often the spectre of double-blind testing, and science in general, is used like a trump card to "win" an argument. The problem is that most who use such don't bother to practice what they preach or to provide any "real" evidence. You have broached a sensitive subject and in retrospect my comments were probably too harsh. What I want to know is what you propose as an evaluation method that is rigourous enough to remove at least some of our tendency toward suggestion. I would even be willing to try it out.

These are not my words, (in fact I hope that the gentleman who wrote it will understand my intentions for posting it here.)

Charlie

****************start of message**************************

Double blind tests need to be organized in a way that is stress reducing, not stress building. Listeners need to have free, sighted PRE-testing of the devices under test with the program material that will be used. They need to have as much time as needed to listen for differences and characterize them so that any difference they believe they are hearing they can be familiar enough with it to recognize it during the double blind experiment. At any time during the double blind experiment they should have the freedom to go back to a sighted casual mode to RE-learn any difference they may feel that are having a hard time identifying during double blind. Also, at times it is good to set up a double blind test that can include extended listening with each device, even days if desired, to more closely simulate the listeners normal approach to evaluating equipment.

Double blind does not have to be a rushed, rigid session that takes all the control away from he listener. I feel the listener should be given all the control possible, short of knowing what device is playing at any given time.

When you do the experiments this way, what you find is that for any REAL differences you can actually detect them easier in the controlled testing than you can with casual sighted testing. I've shown this many times in the past by introducing just noticeable distortion levels and asking listeners to try to notice the change. Most listeners find it very hard to hear a real, just above threshold difference when it is introduced in their system without controls and careful level matched, blind switching. When we go to the controlled method they can identify differences quite easily.

This doesn't seem to sink in with the subjectivists that plug in a new component and immediately wax rhapsodic about the amazing changes they hear.

I try not to get people involved in double-blind "challenges" because those often create a stressful situation that serves no one. They need to be set up as a quest for answers rather than a challenge. That is why it is often much better to train listeners who have no particular stand on the situation rather than use seasoned audiophiles with preconceived notions.

I tend to avoid using double blind tests for vengeful purposes...except when unduly provoked.

It turns out that most uncorrelated differences (ones attributed to, but not related to, different devices under test) are most often heard due to one of four things:

1) Level mismatch

2) Lack of controls relative to listener position, head position or room acoustics constants. Even listening by your self vs. having a friend on each side of you creates a dramatically different acoustic which changes amplitude
responses to levels above audible thresholds.

3) Inherent poor audio memory that us humans have (much worse than most audiophiles know or are willing to admit).

4) Including and related to 3 and most often the culprit: We don't usually compare with a short repeat loop of program material and most instruments actually sound slightly different throughout a song depending on which moment of the song we are comparing to another moment. Music itself is inherently a very poor test signal, from a control standpoint. Most differences can actually be heard much easier with nonmusical test signals but I don't know many audiophiles that want to accept that notion either.

These are the things affecting real perceived differences but not related to the equipment. These don't even include the dreaded imaginary differences due to beliefs about particular equipment or a predetermined attitude that there will be some kind of difference between any two DUTs.

Lot's of room for error folks. Everyone, whether subjectivists or not must not think they are somehow exempt from these illusion creating variables. If you control all of these things you will find that your non blind listening will take on a lot more reality. Double blind can become less necessary to those who use proper care in their controls.

Either use the controls to have more assurance of uncovering reality or don't and just have fun enjoying your system.

Both are valid activities, just be sure to notice which it is that you are doing and don't pretend to be doing one when you are really doing the other.

*****************end of message*****************************

Again, not my composition, but it does have my full endorsement. Enjoy the music! Charlie