Amir and Blind Testing

milpai,

I think he is measuring the wrong thing. Ask your guru to show how he measures a person's emotions. Some folks prefer precision while some prefer musicality.

That's missing the point.  In the case of, say, the Nordost USB cable or PS Audio P12 review, Amirm's measurments indicated no change to the signal that would be audible at all.   Measuring someone's "emotional response" tells you nothing about what's actually happening in reality, in terms of the gear.  I don't give a damn if you have an "emotional response" to a Nordost USB cable because that's you (and likely your imagination).  I want to know if it ACTUALLY does something for the signal so I know what I'm spending my money on.

Some approach high end audio like they do a religion.  Not everyone wants to do that; many of us want actual knowledge as to how the equipment works so we are making informed decisions with our money.

The inherent flaw of any sort of blindfolded A/B testing is when there's unfamiliarity of the test subjects with the items being tested. For example - a blind taste test of Coke vs Pepsi is adversely affected if the test takers aren't cola drinkers. Without a recognizable frame of reference, "best" or "better" is merely a guessing game.

So, playing a piece of music the subjects don't know on a system that is not like their own and asking them to compare that sample to a slightly changed subsequent sample is a waste of time, not a universal truth. Most of us have several pieces of music/performances/albums that we know intimately. If the benchmark used is one of those on our systems (or an equivalent one), then comparative testing has validity, but only then.

Great post!

I concur...

But "Objective measuring tool fetichists" , not Amirm perhaps, but his less enlightened disciples, will claim that sound experience, contrary to any psycho-acoustic/ physical acoustic science fields experience, will come directly and is DECIDED by and from the measured gear specs , not Amirm who is intelligent enough to give only his personal measured numbers, and will SUGGEST that his measures numbers had this meaning or this other one ...But for his disciples this suggestion is a defintive dogma... No listening experiments can contradict it with any value of any kind...Only blind test will defeat SUBJECTIVE biases....And they need to defeat it... But we cannot optimally  tune a SMALL room for ourself  WITHOUT our learned subjective  biases  with  only objective physical acoustic principles ... 😁😊

People are gullible, be it  techno fad "alleged" scientist or those other type of " fetichist who taste their brand name gear" in itself for itself without any objective context to put them at test....The more important context is a room acoustic controlled or not ...

Audio for me is investigation by listening experiments of the acoustic/psycho-acoustic dimension... It is not about a McIntosh or Schiit or Mephisto amplifier specs or price... For sure all piece of gear are different by their design but what we can do to put them at an audiophile level of optimal working ? This is the question...

There is a good amplifier at any price tag....New, old or vintage anyway...

Deception for me in audio is the ignorance of acoustic importance...Bad design will exist even after Amirm measured testing tool bench test , and sometimes good design will exist in spite of his critic with his measuring tool... And anyway what is good in some room may be bad in another room SOMETIMES...

Reality dont emerge from a simplistic formula....

There exist truly bad design but no measures specs are necessary most of the times, a little listening will do...

The inherent flaw of any sort of blindfolded A/B testing is when there’s unfamiliarity of the test subjects with the items being tested. For example - a blind taste test of Coke vs Pepsi is adversely affected if the test takers aren’t cola drinkers. Without a recognizable frame of reference, "best" or "better" is merely a guessing game.

So, playing a piece of music the subjects don’t know on a system that is not like their own and asking them to compare that sample to a slightly changed subsequent sample is a waste of time, not a universal truth. Most of us have several pieces of music/performances/albums that we know intimately. If the benchmark used is one of those on our systems (or an equivalent one), then comparative testing has validity, but only then.

Hello,

not much discussion here about statistics...  well, it's not an exciting subject really.

I thought the point of blind testing was the sample size.  One person is not enough, maybe you need more than 100.  If more than, say, 60% of them opted for A you might be able to say that A is better than B, and if it's 50/50 then that's a pretty good indication that A and B are close to the same thing.  I would venture that when only one person attempts AB you will not get conclusive results unless one of A or B is truly terrible.  

It's hard to get 100 audiophiles into the same room, and you would need good security to prevent fights breaking out.

You are right for sure...

Blind test is most used in the pharma business among others industry...There it is easy to understand why it is very important tool and testing drugs methodology ... Statistical method and blind testing goes hand in hand here...

But in audio if even singular non statistically significant blind test in some case yes can be useful, CLAIMING it must be used systematically on people to DECONSTRUCT their CONTINUOUS experience in meaningless bits is bordering ridiculous when "measuring tool fetichist" warring against "brand name testing subjective fetichist " called it "science"...

Audio is a science where A CORRELATION between subjective experience being maintained must be accounted for with objective tools and methods yes, but not systematically deconstructed by ideological claims...Audio experience is more encompassing than audio industry itself , it is not primarily about gear and tools it is about acoustic/psycho-acoustic and sound/speech/music scientific relation...

Acoustic/psycho-acoustic first and last are sciences in audio with electronic engineering between these two moments of A/P.A...Blind test is a secondary tool at best here, not a primary concern like some fetichist claim for their weak argument about audio which is : We listen and can really listen ONLY what is measured or explained by some set of measures...The rest is deceptive fraud or illusions...

Subjective Perception of tonal timbre for example may be tested by blind test usefully but not negated in itself to exist OBJECTIVELY with the utmost value even if it is a subjective determination... Some people are better than other at this task of timbre tonal perception for sure....Subjective evaluation controlled and correlated with and by objective means is the heart of music/sound experience and science but the reverse is ALSO true, objective means and tools must be subordinated also to the subjective evaluation itself , it is a true TWO WAY correlation process ...Not an ideology...

Hearing is not even understood completely by far today...

It is very revelatory that the two warring fetichist groups battle each other AROUND the gear, focusing on the gear MARKET, and not around acoustic/psycho-acoustic experiments and experience in audio thread...

And like A.I. promotion in some circle is used AGAINST human real INTELIGENCE , which is conceptual creation, in audio digital   industry  some  want and claim  to redefine human hearing experience WITHOUT the need for a  human subject....

This is not science in the two cases but ideological groups inside true science...

Welcome to you here by the way....

Hello,

not much discussion here about statistics... well, it’s not an exciting subject really.

I thought the point of blind testing was the sample size. One person is not enough, maybe you need more than 100. If more than, say, 60% of them opted for A you might be able to say that A is better than B, and if it’s 50/50 then that’s a pretty good indication that A and B are close to the same thing. I would venture that when only one person attempts AB you will not get conclusive results unless one of A or B is truly terrible.

It’s hard to get 100 audiophiles into the same room, and you would need good security to prevent fights breaking out.