Clever Little Clock - high-end audio insanity?

Tbg: I'm still not following your point about sample size, but it sounds like you are saying that it is possible to manipulate statistical methods to prove anything one wants to by the way one chooses the sample size...?

If the effect of the CLC were profound, as some people claim, you could prove this with a very high level of confidence with a very small sample size - there is a 1 in a million probability that someone could correctly guess, purely by chance, whether the CLC was in the house in 21 tries.

A larger sample size is only required if the positive effect is very small, in which case you need a large sample size to show that the small benefit is statistically significant.

I expect that 10 tests would be enough to convince most people that this device does not have much benefit, and a larger number of test would only show with increasing certainty that it has no benefit at all.

Tonnesen, you are actually arguing against statistical significance but are right. If you get a very large random sample of people, there will be statistically significant differences heard between with and without the CLC. But you are wrong that 10 subjects should be enough to convince people that the device has no effect. If you had 10 non-randomly chosen individuals with "good ears'" you might well question whether their hearing a difference can be generalized to all listeners. Similarly, were you to have 10 who doubt the benefit, others might well legitimately question your findings. Even with 10 randomly chosen individuals much would depend on the strength of the treatment effect.

I am not arguing that one should not attempt such tests, but I am arguing that they may not necessitate others heeding them as proof that the CLC does nothing.

Tbg, I'm afraid I hope you're more clear (and relevant, I dare say) with your students...

From what I can tell though, you've got at least one thing basically wrong there:

"I am merely saying that such tests may not be a valid method to prove or disprove whether the CLC does anything"

To me this isn't correct. If enough trials are run, with the clock randomly inserted or removed from the listening environment without the subject seeing which condition it is (I think at least 30 trials would be preferable, which could be divided between 2 or 3 different sessions on different days), its absence or presence should be correctly reported at a rate significantly higher than just 50/50 chance if it's actually doing anything like what the believers maintain. If on the other hand the results hewed pretty close an even 50/50 split, it would be strongly indicative that nothing is audible.

This is a different question than whether whatever the clock may do is 'good' or 'bad', which is a subjective judgement, and not important as to whether the thing can be *detected*. For illustration, I just flipped a coin 30 times, twice. The first set of 30 I got a 17/13 split, the second set a 16/14 split. Whether the splits favored heads or tails is unimportant (as it happens, it was one of each), the relevant point is that my splits only deviated from the mean of 15 by 1 or 2, strongly indicating random outcomes.

Oh, you're talking about random choosing of listeners? I don't think this is necessary, or even desirable. I'd rather limit the listener pool to audiophiles who believe in the audibility of these tweaks.

I have found that even loose testing conditions minimize my ability to hear differences that I previously thought obvious.

Recently, I did some blind testing with a neighbor who has a Krell integrated amp. It drives me nuts, so we dropped in a Modwright 9.0 SE, using the Krell as amp-only. We were both immediately floored. The sound, to me, was soooo much better. He started talking about buying one.

Then, I left it with him for a couple weeks. He did many A/B tests and determined the differences were extremely minor. He blind-tested me and I was fairly ineffective in picking which arrangement was working. He decided the Modwright didn't improve his system and wasn't worth the money.

What does this indicate? Well, my visits to his sound room are again rife with dissatisfaction. The etchy glare is back and I don't really like going over there to listen. Yet, the tests failed to show differences that were obvious in a stress-free environment.

I think this is where testing falls down. How does one know when stress is influencing perception? Further, who wants to subject themselves to testing? It is diametrically opposed to what we normally use our systems for - relaxation and experience.

The idea of a large-sample test does sound promising, and a positive result would be hard to refute. But, it would be nearly impossible to achieve and I'd be suspect of any determination of negativity.

Yeah yeah, making excuses when there isn't even a result yet. . .