@ghdprentice
I would not begin to consider evaluating a single new component without listening to it for a couple months.
I appreciate that you think you need that much time to evaluate a component. But you and I are not similarly situated for many reasons:
1. I understand the full design and architecture of what I am testing. This allows me to focus on what their weakness and strengths are. An example of the former is a powered speaker. These routinely have amplifiers that run out of gas before their drivers do. So I test for that. I am not just shooting in the dark thinking any and all things need to be evaluated.
2. I use measurements which help immensely with #1 above. They show me objectively and reliability where I need to look. If a speaker has a dip in 2 kHz, I use equalization to fill that. I then perform AB testing to determine how audible that is. Measurements are quick. Electronics/tweaks take an afternoon. Speakers take about a day. With that in hand, and knowledge of the product, I am able to make very rapid progress in listening tests.
3. I have tested well over 1000 devices in the last 3 to 4 years. That has enabled me to build methods and systems for fast and reliable comparisons. For example, I have special music tracks that instantly tell me how well a speaker reproduces sub-bass. I know how the competitors to the speakers perform relative to what I am testing.
Audiophiles and "professional reviewers" throw random music at equipment with no aim or direction. So no wonder it takes them so much longer to learn something about the product. At the end they may just be guessing.
4. I am professionally trained critical listener. I also know psychoacoustics and research in this area that says long-term testing is completely unreliable. See the digest of this AES paper:
Here is the punchline there:
The results were that the Long Island group [Audiophile/Take Home Group] was unable to identify the distortion in either of their tests. SMWTMS's listeners also failed the "take home" test scoring 11 correct out of 18 which fails to be significant at the 5% confidence level. However, using the A/B/X test, the SMWTMS not only proved audibility of the distortion within 45 minutes, but they went on to correctly identify a lower amount. The A/B/X test was proven to be more sensitive than long-term listening for this task.
Or if you are more comfortable with video, a complete tutorial in listener training, my ability find small impairments and explanation of above paper:
5. Adaptation. Our brain adapts to its environment. Think of the your computer fan running. After a bit, you forget about it. This is adaptation in play. Same thing happens with say, a speaker that is bright. Listen to it for a while and you adapt and no longer think it is bright. It becomes the "new normal." This is why speakers rank the same in formal studies regardless of the room they are tested in. Your brain learns to listen through the room. From point of view of reviewing, you want to give the true nature of the sound, not what you have adapted to.
Dr. Toole explains this effect very well in his wonderful book:
I could go on but I hope you get the message that I follow the science and research in what I do. What you and other reviewers do is based on lay impressions and what others have told you. You have no proof point that you are creating reliable results. Indeed, research shows as I post earlier, that professional reviewers are terribly unreliable in their assessment of speaker sound.
So you do what you want to do. But unless you can prove your methodology to be right, and better, there is no argument here.
