Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

alexatpos,

 

Again, it’s clear that you have a bias operating that is causing you to place a very negative spin on someone else’s perspective. You are seeing "sins" that are not there.

First, I have not "demanded" you do anything.

I have given my justification for how I personally approach claims about audio gear. Yes, some of the justification entails my not simply believing everything you or any other audiophile claims to hear, because I’m aware of the problem of all too human bias in our perception. But why would any mature adult see that as "rude?"

First of all: do you think all opinions, claims and viewpoints are equally likely and justified? Surely that can’t be, right? If I have the opinion the earth is flat...do you think it is wrong..."ill mannered" even to give reasons why that belief is likely in error?

If someone claims to see X-rays, should that be greeted as equally probable? If someone points out why that is extremely unlikely, why should any reasonable adult greet that as "ill mannered" rather than "trying to be informative?"

Why would we have a strange "anything goes, all things are equally probable on anyone’s opinion" in high end audio? That doesn’t make sense, right?

So if an audiophile is saying "I hear a difference between my expensive USB cable and a properly functioning off-the-shelf USB cable"...someone skeptical of that idea would have to explain WHY they are skeptical. That would include appeal to the nature of how USB/digital signals work and why such claims are unlikely, and also to scientific facts about how our biases can influence our perception, to "perceive" things that aren’t there.

If you take such arguments as being out of bounds and "ill mannered" - because they dare to question the ideas of another audiophile - then you are practicing a religious-like dogmatism, a sort of "questioning anyone's personal view or claimed experience is heresy and not to be countenanced here!"

Why be like that?

My son was involved in a study for a new allergy treatment. It was double blinded placebo controlled. We weren’t allowed to know if he was recieving the active ingrediant or the placebo. Why? Because bias effects - imagining side effects, misinterpreting things etc - are well known variables that need to be controlled for.

Would it have been mature of us to say "How DARE you challenge OUR ability to know what we are experiencing! Your lack of trust is INSULTING!"

That would be just a pure misunderstanding of the real problem the study is designed to address, right?

Yet this is just the type of behavior one often sees here lest anyone voice some skepticism which also has some basis on the facts we really can imagine differences and misperceive things, in audio as anywhere else. It’s greeted, as you are greeting it, as if it’s "rude" and some personal attack.

I’m trying to get you to see this bias because it would be much more fruitful for discussions here if you - and others - stop interpreting any challenge to what you believe, even strongly, as an attack or "rude." It really is that type of reaction that sets these discussions off course. It starts with actually admitting "I Could Be Wrong." Which is always my own assumption and why I’m open to any discussion about why I could be wrong.

Missionary at work …..

@amir_asr, I have a few questions regarding measuring equipment. Genuine questions, not “poking the bear” nonsense. 

 

1. Can unit-to-unit variations significantly affect measurement outcomes?

As we know, all of the individual components in any particular model have specified tolerances, typically ranging between +/- 1% and +/- 20%. Is it possible for a measurable performance difference between two random samples of the same model to result if some of those tolerances stack up one way vs. another, and, if so, can the measured differences be audible?

If measurable differences between two otherwise identical units are possible, then what would be the random sample size necessary to generate a reliable prediction of worst-case, average, and best-case performance?

2. Does a unit’s chain-of custody or provenance impact your confidence in the results of a unit’s test results being truly representative of the model?

Some of the units you have tested were provided directly by the manufacturer or distributor, raising the question of whether or not you received a “golden sample” for your testing. Some of your test units that were submitted by members were discontinued years ago, and many of those were provided by individuals who bought them used in unknown condition. Does either of these situations lead to any additional scrutiny with respect to those units being representative samples?

3. How audible will the difference between an electronic component producing 0.1% THD and one producing 0.001% THD be when played through a transducer generating between 1.0% and 2.0% THD?
4. Will the difference between a 90 dB S/N component and a 120 dB S/N component be audible in a typical listening room with a 30 dB background noise level?

Thanks in advance for your insight.

prof

This is the strong bias that so many bring to these discussions. They view their own opinions on how to evaluate gear as the default - e.g. "The Only Way To Truly Evaluate Gear Is By Listening To It, Like We Do" ...

Hey @prof that’s the logical fallacy known as the strawman argument. In fact, I’ve never seen anyone here make that claim, yet you put quotes around it.

In your next post, you opine:

...it’s clear that you have a bias operating that is causing you to place a very negative spin on someone else’s perspective ...

You commit the sin you blame on others. All your words can’t conceal that.

Missionary at work …..

Ok thyname.   If we are comparing whose position is dogmatic...

I start with a skepticism about my own perception - an acknowledgement of the fallibility of my perception (and reasoning).  Because this is a feature of being human.  Therefore it seems to me that if I REALLY want to be cautious about

a conclusion, I will want to see this factored in to any proposed method of evaluating gear.

And since I don't have all the time in the world, I want some method of guiding where I put my time and money.  So I'll also look to what is technically plausible, to help scale my skepticism and the type of evidence I will prefer.   It's like if someone told me they just bought a 4K TV at Best Buy, I won't bother being skeptical, it's an entirely plausible claim.  If they said they just bought an anti-gravity machine...then based on the inherent improbability, I'll hold off for stronger evidence than their say-so.

Likewise with things like expensive USB cables.  Insofar as I understand how they work, the type of claims often made by audiophiles - and cable companies - for sonic differences are often implausible.  Therefore I would wait for stronger evidence, in the form of measurable differences to the signal and/or listening tests that have controlled for sighted bias.

Of course I could be WRONG about the technical implausibility of the claims.  I am therefore open to arguments for the claims, which I will put against the skeptical case against them to see how they hold up.  I could be WRONG that I can't hear a difference between A or B.  I can always do a blind test to check (I've done several, some positive for differences, others not). I could be WRONG in thinking that someone hearing a sonic difference between their expensive USB cables was due to their imagination.  I am therefore OPEN to evidence I'm wrong - again someone could show measurable changes in a signal, or show they can detect sonic differences in conditions controlling for their sighted bias.

So..my whole approach STARTS with an acknowledgement of my fallibity in perception and reasoning - something all humans share - and tries to account for this in how I approach claims, and also remains perpetually open to arguments or evidence that I'm wrong.  

There is nothing wrong per se about coming to a conclusion on any subject.  We all have to do that to some degree.   What IS a problem is not having an approach that is open to MODIFYING or overturning that conclusion.   Of not being able to say: "this is how I can find out I"m wrong."   THAT is where the true lack of open-mindedness becomes the problem.

This is basically a version of the scientific mindset.

(And it is the view generally shared on ASR )

Now...please explain your alternative to this.

I've been asking audiophiles in the "Listening Is Supreme" camp for a long time:

What Could Show You Are Wrong?

I've never received an answer.   Not once.

By this I mean: If you feel that the most reliable method of evaluating gear is based on whatever you believe you hear..

1. How can you find out when you are in error?

2. What evidence can someone ELSE bring to you, that would overturn your belief?

3.  What would convince you that your very approach to evaluating gear should be re-evaluated?

From what I usually see among such audiophiles...nothing.  So for instance, if an audiophile believes he hears a difference between a Nordost USB cable and a cheap USB cable, my bringing the SAME method of evaluation won't be good enough to challenge their claim.  If I report "I heard no difference" the reply is inevitably "well then you either have a system that can't resolve the difference, or you don't have hearing good enough to resolve the difference.  But They Are There!  I Can Hear Them!"  And...oh...btw "don't bring measurements in to this, we can't measure everything that I Know I Can Hear."

What POSSIBLE evidence can you bring to someone like this, that they are wrong? They've dismissed objective evidence, and will dismiss any subjective evidence as well, because they can ALWAYS say "Well, if others aren't hearing it, too bad for them, but I Know What I Hear."