Non-blind listening tests are flawed, but more time with gear is better than less time. If you could stretch a blind test out into weeks or months, that would be the most accurate way to make the decision.
Why blind listening tests are flawed
This may sound like pure flame war bait - but here it is anyway. Since rebuilding my system from scratch, and auditioning everything from preamps to amps to dacs to interconnects to speaker cable etc, it seems clearer than ever.
I notice that I get easily fooled between bad and great sounding gear during blind auditions. Most would say "That should tell you that the quality of the gear is closer than you thought. Trust it".
But it's the process of blind listening tests that's causing the confusion, not a case of what I prefer to believe or justify to myself. And I think I know why it happens.
Understanding the sound of audio gear is process of accumulated memories. You can listen to say new speakers for weeks and love them until you start hearing something that bothers you until you can't stand them anymore.
Subconsciously you're building a library of impressions that continues to fill in the blanks of the overall sound. When all the holes are filled - you finally have a very clear grasp of the sonic signature. But we know that doesn't happen overnight.
This explains why many times you'll love how something sounds until you don't anymore? Anyone experience that? I have - with all 3 B&W speakers upgrades I've made in my life just to name a few.
Swapping out gear short term for blind listening tests is therefore counter productive for accurately understanding the characteristics of any particular piece or system because it causes discontinuity with impression accumulation and becomes subtractive rather than additive. Confusion becomes the guaranteed outcome instead of clarity. In fact it's a systematic unlearning of the sound characteristics as the impression accumulation is randomized. Wish I could think of a simpler way of saying that..
Ok this is getting even further out there but: Also I believe that when you're listening while looking at equipment there are certain anchors that also accumulate. You may hear a high hat that sounds shimmering and subconsciously that impression is associated with some metallic color or other visual aspect of the equipment you happen to be watching or remember.
By looking at (or even mentally picturing) your equipment over time you have an immediate association with its' sound. Sounds strange, but I've noticed this happening myself - and I have no doubt it speeds up the process of getting a peg on the overall sound character.
Obviously blind tests would void that aspect too resulting in less information rather than more for comparison.
Anyone agree with this, because I don't remember hearing this POV before. But I'm sure many others that have stated this because, of course, it happens to be true. ;
I notice that I get easily fooled between bad and great sounding gear during blind auditions. Most would say "That should tell you that the quality of the gear is closer than you thought. Trust it".
But it's the process of blind listening tests that's causing the confusion, not a case of what I prefer to believe or justify to myself. And I think I know why it happens.
Understanding the sound of audio gear is process of accumulated memories. You can listen to say new speakers for weeks and love them until you start hearing something that bothers you until you can't stand them anymore.
Subconsciously you're building a library of impressions that continues to fill in the blanks of the overall sound. When all the holes are filled - you finally have a very clear grasp of the sonic signature. But we know that doesn't happen overnight.
This explains why many times you'll love how something sounds until you don't anymore? Anyone experience that? I have - with all 3 B&W speakers upgrades I've made in my life just to name a few.
Swapping out gear short term for blind listening tests is therefore counter productive for accurately understanding the characteristics of any particular piece or system because it causes discontinuity with impression accumulation and becomes subtractive rather than additive. Confusion becomes the guaranteed outcome instead of clarity. In fact it's a systematic unlearning of the sound characteristics as the impression accumulation is randomized. Wish I could think of a simpler way of saying that..
Ok this is getting even further out there but: Also I believe that when you're listening while looking at equipment there are certain anchors that also accumulate. You may hear a high hat that sounds shimmering and subconsciously that impression is associated with some metallic color or other visual aspect of the equipment you happen to be watching or remember.
By looking at (or even mentally picturing) your equipment over time you have an immediate association with its' sound. Sounds strange, but I've noticed this happening myself - and I have no doubt it speeds up the process of getting a peg on the overall sound character.
Obviously blind tests would void that aspect too resulting in less information rather than more for comparison.
Anyone agree with this, because I don't remember hearing this POV before. But I'm sure many others that have stated this because, of course, it happens to be true. ;
- ...
- 46 posts total
A customer of ours has been using a microphone to measure the results in his system as a result of minor changes he has done. He's on to something- you can clearly see how distortion has changed due to his changes. http://www.audioasylum.com/forums/otl/messages/3/36516.html Funny- the objectivist camp says that if you hear something but can't measure it, its not real. Yet I never see them use this technique. Instead they might test one piece of equipment on the bench rather than in the system. Its not scientific! As a result, I don't seem them having any numbers at all to support their position- they are the pot calling the kettle black! |
"Funny- the objectivist camp says that if you hear something but can't measure it, its not real." I would say it may or may not be real and there is no objective way to determine for certain. Maybe if teh test is done exactly the same way in a controlled environment for a large enough sample size, the results might turn out to be significant statistically. That's different than saying its not real unless it can be measured. There is no real basis to say any single test case is indicative of reality or not. A pure stand either way is just wrong. |
When I used to sell gear years ago, blind listening tests for customers, though flawed for reasons the OP described, were the best tool available to help a customer decide what they liked best, AT THE TIME. But Time Always tells. From a buyers perspective, the best approach is to train your ears by listening to a lot of music as much as possible, as many ways as possible, then use that as the reference for determining where things stand with your setup. Then tweak and tune as needed to hit your target. Meanwhile, keep on listening along the way to as much as possible as many different ways as possible to know how things can and do really sound. At some point you will know the target when you hear it, and it probably will not even change very much, and only then will you be in a position to bring the hunt to an end. SO the key is to know how your gear sounds and compares to everything else. BLind a/b listening tests are not needed. |
- 46 posts total