I see the issue with ABX blind testing

My issue with having people over for blind testing is all the stumbling around and knocking things over. Otherwise it's great.

soix

8,549 posts

04-11-2024 at 08:24pm

 

“And this has been demonstrated to take less time than many audio gurus assume. If comparisons are not a literal flip-switch, subjects can be quick to lose, alter or invent, context. A “trust your ears” stance is necessarily deaf to this ubiquitous limitation, and I think that’s fine so long as the choice is not professed to be useful for everyone’s case.”

 

@benanders Lots of things have been “demonstrated” one way or another but doesn’t make something “ubiquitous.” I’ve heard consistent differences manually switching between many cables and did not need instantaneous switching to hear it, and these difference were consistent, clear, and repeatable. Some people, apparently like yourself, don’t possess the ability to do this so can’t trust their own ears, but it’s a very useful method for those of us who can.

Heyya @soix certain biological “limitations” in human hearing x associated cognitive functions certainly have demonstrated ubiquity. Despite that, there’s nothing necessarily wrong with you or anyone else trusting your respective perceptions in music kit. Professing those perceptions to be “consistent, clear, and repeatable” for others is, however, either in honest error or purposefully misleading. You need a proper assessment to claim broad applicability of perception(s), and typing disparaging speculation about someone’s hearing and/or audio kit doesn’t supplant that need, no matter how many times you do the latter. 😉

audphile1

4,068 posts

04-11-2024 at 09:19pm

 

@soix your arguments are falling on deaf ears.

@audphile1 WHAT??

Lol

Professing those perceptions to be “consistent, clear, and repeatable” for others is, however, either in honest error or purposefully misleading.

@benanders So, you’re saying I’m not hearing what I’m hearing despite hearing it consistently after multiple back-and-forth comparisons?  It’s really sad some people need studies to tell them what they can and can’t hear.  I’d submit it’s misleading (and arrogant) for you to maintain that using your ears is an error and misleading based on some study somewhere.  Pretty sure most people here have been able to discern differences between two products, but I guess you need to be told what you can hear rather than being able to objectively judge something for yourself.  Sad.  Deaf ears indeed.

A fundamental property of human perception is that it is variable. The same person experiencing the same thing on different occasions will have a slightly different experience of that thing. Different people experiencing the same thing at the same time will also have slightly different experiences of that thing. That is why measurement of psychological phenomena typically makes use of samples of data. If each person's experience is measured using a psychologically valid scale you will get a distribution of measurements with an average value and a degree of variability (e.g., standard deviation or statistical error). This enables you to say whether two distributions of data are different from one another (or not). 

So, invite a dozen buddies over to listen to each cable (not telling them which is which) and have each of them rate each cable using a psychologically valid metric (a 7-point scale anchored with the labels Like Extremely, Like Somewhat, Like Slightly, No Opinion, Dislike Slightly, Dislike Somewhat, Dislike Extremely would suffice) and you will get an average and a standard deviation for each cable. Calculate the Mean and 95% Confidence Interval for each distribution and if the two don't overlap, well, then, pick the cable with the highest mean Liking.

You can see that this takes a bit of work, but that is what is required for a  psychologically valid answer. I have actually done such research for several corporations at which I have worked. Same thing applies for visual and tactile experiences.