Clever Little Clock - high-end audio insanity?

Tonnesen, you are actually arguing against statistical significance but are right. If you get a very large random sample of people, there will be statistically significant differences heard between with and without the CLC. But you are wrong that 10 subjects should be enough to convince people that the device has no effect. If you had 10 non-randomly chosen individuals with "good ears'" you might well question whether their hearing a difference can be generalized to all listeners. Similarly, were you to have 10 who doubt the benefit, others might well legitimately question your findings. Even with 10 randomly chosen individuals much would depend on the strength of the treatment effect.

I am not arguing that one should not attempt such tests, but I am arguing that they may not necessitate others heeding them as proof that the CLC does nothing.

Tbg, I'm afraid I hope you're more clear (and relevant, I dare say) with your students...

From what I can tell though, you've got at least one thing basically wrong there:

"I am merely saying that such tests may not be a valid method to prove or disprove whether the CLC does anything"

To me this isn't correct. If enough trials are run, with the clock randomly inserted or removed from the listening environment without the subject seeing which condition it is (I think at least 30 trials would be preferable, which could be divided between 2 or 3 different sessions on different days), its absence or presence should be correctly reported at a rate significantly higher than just 50/50 chance if it's actually doing anything like what the believers maintain. If on the other hand the results hewed pretty close an even 50/50 split, it would be strongly indicative that nothing is audible.

This is a different question than whether whatever the clock may do is 'good' or 'bad', which is a subjective judgement, and not important as to whether the thing can be *detected*. For illustration, I just flipped a coin 30 times, twice. The first set of 30 I got a 17/13 split, the second set a 16/14 split. Whether the splits favored heads or tails is unimportant (as it happens, it was one of each), the relevant point is that my splits only deviated from the mean of 15 by 1 or 2, strongly indicating random outcomes.

Oh, you're talking about random choosing of listeners? I don't think this is necessary, or even desirable. I'd rather limit the listener pool to audiophiles who believe in the audibility of these tweaks.

I have found that even loose testing conditions minimize my ability to hear differences that I previously thought obvious.

Recently, I did some blind testing with a neighbor who has a Krell integrated amp. It drives me nuts, so we dropped in a Modwright 9.0 SE, using the Krell as amp-only. We were both immediately floored. The sound, to me, was soooo much better. He started talking about buying one.

Then, I left it with him for a couple weeks. He did many A/B tests and determined the differences were extremely minor. He blind-tested me and I was fairly ineffective in picking which arrangement was working. He decided the Modwright didn't improve his system and wasn't worth the money.

What does this indicate? Well, my visits to his sound room are again rife with dissatisfaction. The etchy glare is back and I don't really like going over there to listen. Yet, the tests failed to show differences that were obvious in a stress-free environment.

I think this is where testing falls down. How does one know when stress is influencing perception? Further, who wants to subject themselves to testing? It is diametrically opposed to what we normally use our systems for - relaxation and experience.

The idea of a large-sample test does sound promising, and a positive result would be hard to refute. But, it would be nearly impossible to achieve and I'd be suspect of any determination of negativity.

Yeah yeah, making excuses when there isn't even a result yet. . .

Miklorsmith: I agree (and have detailed before) that there can be problems with formal testing methodologies as applied to subjective auditioning. I do think some kinds of testing can introduce a "confusion factor" that may actually serve to artificially raise the floor for perceivability of low-level differences. And I think it's to a large extent possible to ameliorate biasing effects due to external factors without resorting to blind tests, though it can take repetition over time and a certain self-questioning mindset (that I'm learning a lot of audiophiles seem to lack). As for how test conditions might significantly differ from normal use conditions, this can be good or bad -- I don't listen to music for enjoyment by performing rapid A/B comparisons, but doing them can really help nail down (or dismiss) some elusive observations concerning gear.

But, when faced with a product or claim that appears to carry all the hallmarks of snakeoil, and audiophiles buying into it using the most casual and fallible auditioning methods, I don't think it's inappropriate to call for some demonstrable degree of rigor to be brought to bear. I also think that experiences like the one you relate above are valuable for putting things in proper perspective every once in a while.