In defense of ABX testing

Zd542 wrote,

"For years I've been asking them to show me the tests."

I suspect maybe you've been asking the wrong people. Nobody is denying the difficulty in devising tests that work like they're supposed to, I.e. demonstrate whether A is better than B or whatever. Or whether some newfangled device is a fraud. Nobody is denying the existence of placebo effect or expectation bias or it's ugly sibling the reverse expectation bias or any other such psychological effects. But to declare that there are no proper tests is a little bit inaccurate.

"Nobody is denying the existence of placebo effect or expectation bias or it's ugly sibling the reverse expectation bias or any other such psychological effects. But to declare that there are no proper tests is a little bit inaccurate."

No, its not. If you take my quote that you reference, "For years I've been asking them to show me the tests.", and put it in context with the rest of my statement:

"If you're right, and you really know what you are talking about, pick 2 products, conduct the test, and report your findings in a way so the rest of us can try it for ourselves. That's how a real scientist would do it."

we get a different picture. To me, if you wanted to conduct a scientificly valid listening test to compare 2 audio products, I can't see you going wrong doing it the above way. Not only that, if you read through the thread that the OP referenced, you'll see that, at least some of them, do agree with me. So again, it all boils down to the same exact thing. Show me the test's. If me declaring that there are no proper tests done is a little bit inaccurate, by all means, show me. I'll settle for just 1. And just to be clear, I really have been asking for years. I'm 100% serious. If you really want to, you can check my old AG threads.

Sorry, but before I forget; "I suspect maybe you've been asking the wrong people.". Maybe you're right about that. Who do I ask? I mean if I don't get results at a place like Hydrogen, where its their mission statement to go by science, and will actually censor threads if they don't contain content approved by the moderators, then who do I ask? If you really take a step back and look at this whole issue, the people who talk about doing these listening tests the most, avoid it like the plague. It doesn't even make sense.

I agree with the notion of Science! being applied as a tool for understanding how different equipment influences my perceptions, but I'm not convinced ABX testing is the right way to go about it.

Even if a test population of reasonable size is acquired, large enough to smooth out the differences in loudness perception in the subjects, you'd still only have a result for that particular set of equipment. Perhaps a statistically significant percentage of the subject group could identify MP3 versus APE, but once you swap out a cable or change the room temperature, confidence in the results degrade.

If one wants to apply science here, there first has to be a hypothesis that can be falsified through measured experiment. For example, "all conducting wire measures the exact same frequency response curve with white noise at 90db, regardless of the transducer". This is probably trivial to falsify, but is on the same continuum of the notion that high priced cables are "better" than lamp cord.

If you want to apply science to that or a similar or even more refined question, super. But an ABX test isn't going to get you a definitive result.

This dead horse has been flogged for so long that it's turning me into a vegetarian.