Clever Little Clock - high-end audio insanity?

haha, i cant believe that this thread is still going on.

You're welcome Bigjoe!

Zaikesman and Tonnesen, as you know, tests of statistical significance are sensitive to how big the sample is. With a sample of 25,000, any relationship will prove statistically significant. Since we are all too willing to say that a relationship that is statistically significant is also significant, we are in danger of saying as you think I am saying that you can prove anything with statistical significance tests. These tests were developed to answer the simple question of whether an unusual random sample from a population where there was no difference could have given us sample results where there is a difference.

In the tests that you both propose as to whether subject hear or don't hear the CLC is present, a large sample of say 10,000 would achieve statistical significance even were there no difference, although I would not predict in which direction, such as whether the CLS helped or hurt.

I am not being anti-science or anti-logic, I am merely saying that such tests may not be a valid method to prove or disprove whether the CLC does anything. I am also saying that those who claim it does nothing cannot claim the high ground by saying that those hearing a difference are delusional as those hearing no difference may also be affected by prior conceptions.

Tbg: I'm still not following your point about sample size, but it sounds like you are saying that it is possible to manipulate statistical methods to prove anything one wants to by the way one chooses the sample size...?

If the effect of the CLC were profound, as some people claim, you could prove this with a very high level of confidence with a very small sample size - there is a 1 in a million probability that someone could correctly guess, purely by chance, whether the CLC was in the house in 21 tries.

A larger sample size is only required if the positive effect is very small, in which case you need a large sample size to show that the small benefit is statistically significant.

I expect that 10 tests would be enough to convince most people that this device does not have much benefit, and a larger number of test would only show with increasing certainty that it has no benefit at all.

Tonnesen, you are actually arguing against statistical significance but are right. If you get a very large random sample of people, there will be statistically significant differences heard between with and without the CLC. But you are wrong that 10 subjects should be enough to convince people that the device has no effect. If you had 10 non-randomly chosen individuals with "good ears'" you might well question whether their hearing a difference can be generalized to all listeners. Similarly, were you to have 10 who doubt the benefit, others might well legitimately question your findings. Even with 10 randomly chosen individuals much would depend on the strength of the treatment effect.

I am not arguing that one should not attempt such tests, but I am arguing that they may not necessitate others heeding them as proof that the CLC does nothing.