Blind Testing is Dead - Long live My Wallet testing.


Hi Everyone,

I was seeing some discussions around cables, and reading other discussions about A'gon members asking for opinions on different alternatives for hooking up a DAC, or TV sound, or whatever, and it made me think of this.


I want to tie a few things together:

  • Most technical measurements consumers read were defined by the 1970s. It is fair to describe them as stagnant.
  • The cost to benefit ratio of a lot of products can vary a great deal.
  • I hear things I can't yet measure in cables and crossover components.
  • I like measurements. 
  • Someday measurements commonly discussed among consumers will improve and better tie our values to technology.

A lot has been made about double blind testing, and a lot of readers rely on taste masters (web sites, magazines and social media) and whether in fact these taste masters can hear anything at all. Reminds me a lot of blind testing of wines, or an article I read recently about how much super rare whiskey is fake.


When deciding on a bit of kit, I could not care less about double blind testing. I care about :

  • What audible value can I perceive?
  • Is the price proportional to that value?
  • Is my money better spent on a vacation or liquor?

We should also note that I'm a bit of an iconoclast. Most consumers also care about:

  • Brand recognition
  • Style
  • Perception of modernity (is it cutting edge no one else has)
  • Perception of construction (how much does it weigh, how is it packaged)
  • Ability to create envy.
  • Price ( if it's too inexpensive, it can't be good! )


What is my message then? My message is that this is all cute, like reading about movies or books or music shows, but in the end, it's my wallet, no one else's. John Atkinson is not buying my speakers for me. I am. My hard work creates value which I use some of (sometimes too much) to buy audio related products. The more you detach yourself from brands, costs and worries about measurements the more frugal, and happier  you will be.


Best,

Erik


erik_squires

@clearthink, if these esteemed reviewers with all of their years and years of listening and critically evaluating equipment cannot reach a clear evaluation through blind testing then we are forced to conclude that either the differences are so subliminal as to hardly be worth the bother or there aren't any. 

Let's face it, subliminal differences are of little use to most of us audiophiles, but blind evaluation must always reveal major ones, ones that we can hear 365 days a year, ones we are all interested in. I can't speak for others but I have no interest in pursuing subtle subliminal improvements when I am looking to upgrade. Many years ago I started with a Rega 3 and later upgraded to an LP12. Was it an upgrade? Yes. Was it worth the expense and trouble? No, but that's another story.

I agree that their opinion of reviewers is far from being the final word but its beginning to look as if that opinion is almost entirely worthless. Little more than subjective rambling in most cases. Unfortunately many still rely upon reviews even in 2019 instead of regarding them as no more than light, but frivolous entertainment.

HiVyNyws channel is one of the very few left that doesn't pull his punches. But then he's reviewing vinyl where real (as distinct from imagined) differences are easy to demonstrate.
https://youtu.be/E4G7hSUQoO

In cases where home auditioning is impossible perhaps we need to go back to evaluating and buying gear on specifications once more? If you know how to read them they can tell you at a glance a lot more than some lilly livered critic ever will.

millercarbon
"
Components should have stickers like wine: 92 points- rich, fully developed acoustic space, strong wood notes but not too brassy"

Those who are knowledgeable, educated, and experienced do not need others to tell them how equipment sounds whether it be through labels, pronouncements on the Internet, or the claims, beliefs, and convictions of other "audiophiles." 

cd318
"
if these esteemed reviewers with all of their years and years of listening and critically evaluating equipment cannot reach a clear evaluation through blind testing then we are forced to conclude that either the differences are so subliminal as to hardly be worth the bother or there aren't any."

I am in no way forced, compelled, or inclined to reach this conclusion that you state because I am not subject to the two logical fallacies you have committed hear in one sentence!

" subliminal differences are of little use to most of us audiophiles, but blind evaluation must always reveal major ones"

This is an opinion stated as fact yet another logical fallacy!

" In cases where home auditioning is impossible perhaps we need to go back to evaluating and buying gear on specifications once more? If you know how to read them they can tell you at a glance a lot more than some lilly livered critic ever will"

You cannot judge a book by it's cover or a Music Reproduction System component or accessory based on measurements alone this has been repeatedly, reliably, and consistently shown, demonstrated, and revealed.

Truel. true. the wallet delivers the final verdict.

but, blind listening is not dead!

trust me on that one.

my recent visit to an audio Show offered a few ways to examine, listen, investigate, and or ‘value/appraise’ its various exhibitions.

I could ask right off what is being demonstrated front to back, top to bottom, then get quite attentive and decide how to judge each demo.
 
I chose only to find out the brand of the speakers and then, which amp (s) were driving them.

if the demo was one that reached out and grabbed me, then I would get more details.

entering every room was for me, a real blind audition.

of all the folks here esthetics should not be much of or at all, a biggie for me when deciding between pieces of audio gear.

but it is….

this is mostly with racks and speakers, not so much with the electronics per se.

but there is only one way to note the influence or input of any device within an audio system and that is to know the system itself prior to exchanging ANY facet within it.

otherwise, the only known value of any outfit which can be remarked upon sanely will be the system’s loud speakers.

all other anecdotal feedback, input, articles, press, blogs are incidentals, asides, humourous or otherwise are instances of time well or poorly wasted.

the only thing I extract from published notes on this or that device or accessory might be to see what ever commonalities are contained in the various judgements.

its plain testy and difficult to perfectly setup ‘blind’ testing of a particular device to begin with, though with some effort and within some reasonableness it can be done so at least the broadest strokes of change can be unearthed by the listeners.. .

Many here have made a choice based mostly on faith when procuring their next piece for their stereo system.

so then it is quite true that at times even ‘blind wallets’ will buy the next gizmo.