What's your process for comparing new gear, cables, etc.?


It's a staple of many if not most posts to compare auditory experiences and attribute them to different factors — larger components (room, speaker, amp, dac, etc.) and the rest (speaker wire, cables, power) etc. This is how we choose new gear or compare what we already have.

Given the number of components and how short auditory sensory memory is, any comparison should change the fewest variables possible, as quickly as possible. (And auditory memory is short, even for simpler sounds. We compare using complex sounds and want to judge complex dynamic effect — soundstage, tonality at various frequencies, overall character or musicality, etc.) Doing things quickly is a challenge with tube amps, which must be shut down properly in order to swap things out. Then, they must be turned on and warmed up a bit.

I'm curious how people conduct their comparisons given whatever factors they contend with. Do you take notes? Have a standard vocabulary (e.g. the one in Harley's book)? Use a checklist? Have certain test tracks that you have virtually memorized? And so on. I'd like to know what works for you.

Most of the time, I'm just listening to music and enjoying it. But when I do want to add gear or make a change, it's natural for a critical comparison to call for some kind of procedure. I'm still trying to figure out what procedure can provide reliable, practical information. When my procedure seems too random or complicated, I feel a bit absurd — like I'm just doing kabuki-science! 


128x128hilde45
I appreciate the thoughtful responses. I will not respond to insults or condescension. I phrased my question carefully. And yet...

So, to the thoughtful responders: I will think about what’s been said and respond when I have a moment.
@tommic601 -- the specific mentions of equipment and techniques is very helpful, as is the encouragement.

In the meantime, I found this interesting piece: http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf

I have not read it carefully or fully, so I am not endorsing it, but clearly auditory memory is a complex psychological subject area.
I’ll bite, and I’ll share what I’m planning to do over next couple of months.

I’m waiting to take delivery from local dealer on streamer/dac that will complete a new second system, which I have yet to assemble. All of the components (speakers, power amp, dac/streamer, power conditioner) are new, as are all of the cables. I’ll burn everything in for a week or so and listen to it and come to terms with what I have.

Nothing too interesting there.

But as to my planned experiment, I’ve also (just for fun and experimentation purposes), acquired three new sets of alternative cabling, each consisting of 3 pieces (power cord for streamer/dac, interconnect between streamer/dac and power amp, and speaker cables). [power cable to amp and sub will remain constant thru experiment, as I’ve read that power cord to source is most important].

The three alternative sets are at different cost levels, each less than what I’m planning to have as my true set with this system, that I refer to in para 2.

The three sets are:
at the low end, cost-wise: AmazonBasics speaker and interconnect, and W Audio power cable (also available on Amazon) - msrp around $60 total.

in the middle, cost-wise: Pangea power cable and Blue Jeans IC and speaker cables - msrp around $180.

finally: Audioquest power, IC and speaker cables - msrp around $575.

I will break each of these in also for about a week, together as a set, and listen to determine whether and how much I detect differences in these three alternative sets of cables, and how they compare to the set I selected for the system.

I realize this is changing three things at a time, instead of one, and if I really wanted to go crazy I could spend the rest of my life looking at the permutations and combinations of everything involved, but .... three sets will take enough time and exhaust my experimentation curiosity given the limited amount of time in a day I have to spend on this, haha.

Thereafter, assuming the preselected set survives, I’ll keep the three alternative sets to let extended family members or friends just starting out experiment with to see whether they hear differences in their systems and whether cable upgrades might make sense for them based on what they hear. Probably they’ll say, "you kidding! why would I go to that much work??! - you’re nuts!" But on here it’s all good : )
Thanks @kren.
There is a new You Tube channel on audio. Notice he says, "Short term auditory memory has been shown to be very unreliable, even the order in which you switch components makes a difference."

His reply to my question was interesting. Sharing:

Tarun A British Audiophile
"Hi David, that is an excellent question. You have to try and eliminate as many variables as possible. Here is my process...

The room: I need to understand the acoustic characteristics of my room so that I can develop an ability to listen “through” the room. If I am evaluating speakers, I need to spend quite a bit of time experimenting with positioning to try and get the best out of them. I have a listening room on the acoustically lively side. It helps to have another room that has more damping just to double check my conclusions. Although, with experience this becomes seldomly necessary.

Partnering equipment: I use a minimum of 3 DACs, 3 amplifiers and 3 speakers to evaluate a new component. I have owned these components for some time so instinctively recognise their sound characteristics. Although, I mainly listen through my main system if I am evaluating a new piece of equipment, it is important to try any new component with different combinations so that I can determine how it behaves and what might be suitable partnering equipment. For example, my Exposure Pre/Monos have a big, warm, rich sound where as my Hegel H160 is much leaner and more clinical in its presentation. My Audiolab M-PWR is a compact 40 watt amp that shows if a speaker is difficult to drive. Listening material: Having test tracks that I know inside out helps because I know instinctively how it should sound ordinarily and can more easily identify changes. Tracks are selected based on their ability to highlight a specific aspect of a products performance. For example, I have certain tracks that I listen to for evaluating female vocals, soundstage, imaging, transient response, etc. Most of them are great quality recordings but not every track because it is also import to know how forgiving a component may be of less than perfect recordings.

AB testing: I resist the temptation to switch components after minutes or hours of listening. Short term auditory memory has been shown to be very unreliable, even the order in which you switch components makes a difference. Have you noticed that when manufacturers do this they always start with the cheapest component and then go to the more expensive one and not the other way around. There is a very good reason for this. The second time you hear something new, it will sound better even if you haven’t changed anything. IMHO the only way to evaluate a component is to listen to it for days, making notes that you refer to and update through the process. Product burn-in may be a hotly contested debate but human burn-in is definitely an element to be considered when evaluating any new component. Give yourself time to adjust to how a product sounds, allow yourself to slowly form an impression, once you feel comfortable, only then, change a component. Is this process scientific? Absolutely not but I know of no better way to evaluate new components.

Thank you for the question. I may do a video on this topic."
I find that listening over the long term is a much better way for me to evaluate the differences that components make.  

That's not to say that doing A/B testing doesn't point out differences, it does.  But what it points out is that there is a difference, not so much what the difference is. 

Sometimes it's easy to define the differences you hear doing a quick A/B test, but for me at least, it takes extended listening to pick up on the more subtle nuances that a change in gear makes and what those nuances are.
Listening covers a wide range of subjects and reasons. The only thing wider is the range of sonic characteristics we are able to hear. This goes well beyond easily measured things like the volume and frequency response so many obsess over, and into things like imaging, depth, focus, air, inner detail, on and on. The list is endless.

The biggest stumbling block I have seen, and it is prevalent, so depressingly prevalent, is the compulsion to doubt and second guess and attempt to explain away things you do in fact hear.

When I first started trying to listen for some of these things it was extremely frustrating. In hindsight the two things that held me back the most were the lack of vocabulary, the ability to understand and put into words the differences I was in fact hearing, and this false advice about the necessity of precisely matching everything. 

So what would happen, and this happened over and over again, I would go to some store and compare CDP or whatever and find one I thought sounded better. I was sure it sounded better, just not in any way I could put into words. So I would get my wife to come and listen. And always, every single time, she would indeed have the same preference.

And no, for you guys who think you're so smart, I know all the stories, every single one of them. More than you I bet. So I was careful to not say anything to let on which one I liked. Just I think there's a difference but I can't believe it and want to be sure so would you mind? And every time she would say well I like this one. Why? It sounds more expensive. More, quality. That's it. We knew. We heard it. We agreed. We just had no way of putting it into words.

One of the things I always truly believed even after a lot of other "common wisdom" had been proven false, was you had to listen at a certain volume level. 

Then one day it dawned on me. The one guy who had taught me the most, the best listener I ever came across, who always had the best components and knew every detail of how they sounded, Stewart Marcantoni, this guy never once listened above probably about 85 dB. I would always have to turn it up. When he came over I would always respect this and play a little lower volume for him than anyone else. 

I thought for a long time he was just protecting his ears. More and more it bugged me, how he is able to hear so much detail so well even at such a low volume. Until one day I asked and he told me that's part of it, saving his ears, but really you do not need to listen so loud to hear just fine. Around that same time I learned you do not even need to play the same recording over and over again. Or even twice. When you know the system and are a good enough listener none of this is necessary.

People find it good sport to insult and make fun of me. I am fair game. Everyone else can make fun of my system. I can't do the same. I get that. Doesn't change the fact the stories are true. That guy at CES, he really did hear a system for the first time and he really could tell on one hearing what was wrong and they really did fix it. 

Didn't need no matched volume. Didn't even need no A/B. For sure did not need any double-blind. All he needed was his ears. And listening skills developed to a sublime degree from decades of practice.

Anyone serious about learning, this is how you go about it. You can start with whatever crutches you need. Match volume. Play the same tracks until you never want to hear them again. Whatever floats your boat. Just don't come around here telling me its some kind of requirement. Its not. Its just some mental roadblocks you put in your own way. You put them there. You can remove them. Or not. Just remember they are your roadblocks. Not mine.