What's your process for comparing new gear, cables, etc.?


It's a staple of many if not most posts to compare auditory experiences and attribute them to different factors — larger components (room, speaker, amp, dac, etc.) and the rest (speaker wire, cables, power) etc. This is how we choose new gear or compare what we already have.

Given the number of components and how short auditory sensory memory is, any comparison should change the fewest variables possible, as quickly as possible. (And auditory memory is short, even for simpler sounds. We compare using complex sounds and want to judge complex dynamic effect — soundstage, tonality at various frequencies, overall character or musicality, etc.) Doing things quickly is a challenge with tube amps, which must be shut down properly in order to swap things out. Then, they must be turned on and warmed up a bit.

I'm curious how people conduct their comparisons given whatever factors they contend with. Do you take notes? Have a standard vocabulary (e.g. the one in Harley's book)? Use a checklist? Have certain test tracks that you have virtually memorized? And so on. I'd like to know what works for you.

Most of the time, I'm just listening to music and enjoying it. But when I do want to add gear or make a change, it's natural for a critical comparison to call for some kind of procedure. I'm still trying to figure out what procedure can provide reliable, practical information. When my procedure seems too random or complicated, I feel a bit absurd — like I'm just doing kabuki-science! 


128x128hilde45
On single variable changes with cables it is best to gently demate/mate the baseline connects, allow some settle in time listen then make switch to the variable cable. One reason why I like Audioquest DBS system is that the battery is always forming the dialectic- this greatly speeds up a cable A/B.

its also a good practice to get on a demate/clean/mate maintain cycle for the whole system 
I appreciate the thoughtful responses. I will not respond to insults or condescension. I phrased my question carefully. And yet...

So, to the thoughtful responders: I will think about what’s been said and respond when I have a moment.
@tommic601 -- the specific mentions of equipment and techniques is very helpful, as is the encouragement.

In the meantime, I found this interesting piece: http://www.acourate.com/Download/BiasesInModernAudioQualityListeningTests.pdf

I have not read it carefully or fully, so I am not endorsing it, but clearly auditory memory is a complex psychological subject area.
I’ll bite, and I’ll share what I’m planning to do over next couple of months.

I’m waiting to take delivery from local dealer on streamer/dac that will complete a new second system, which I have yet to assemble. All of the components (speakers, power amp, dac/streamer, power conditioner) are new, as are all of the cables. I’ll burn everything in for a week or so and listen to it and come to terms with what I have.

Nothing too interesting there.

But as to my planned experiment, I’ve also (just for fun and experimentation purposes), acquired three new sets of alternative cabling, each consisting of 3 pieces (power cord for streamer/dac, interconnect between streamer/dac and power amp, and speaker cables). [power cable to amp and sub will remain constant thru experiment, as I’ve read that power cord to source is most important].

The three alternative sets are at different cost levels, each less than what I’m planning to have as my true set with this system, that I refer to in para 2.

The three sets are:
at the low end, cost-wise: AmazonBasics speaker and interconnect, and W Audio power cable (also available on Amazon) - msrp around $60 total.

in the middle, cost-wise: Pangea power cable and Blue Jeans IC and speaker cables - msrp around $180.

finally: Audioquest power, IC and speaker cables - msrp around $575.

I will break each of these in also for about a week, together as a set, and listen to determine whether and how much I detect differences in these three alternative sets of cables, and how they compare to the set I selected for the system.

I realize this is changing three things at a time, instead of one, and if I really wanted to go crazy I could spend the rest of my life looking at the permutations and combinations of everything involved, but .... three sets will take enough time and exhaust my experimentation curiosity given the limited amount of time in a day I have to spend on this, haha.

Thereafter, assuming the preselected set survives, I’ll keep the three alternative sets to let extended family members or friends just starting out experiment with to see whether they hear differences in their systems and whether cable upgrades might make sense for them based on what they hear. Probably they’ll say, "you kidding! why would I go to that much work??! - you’re nuts!" But on here it’s all good : )
Thanks @kren.
There is a new You Tube channel on audio. Notice he says, "Short term auditory memory has been shown to be very unreliable, even the order in which you switch components makes a difference."

His reply to my question was interesting. Sharing:

Tarun A British Audiophile
"Hi David, that is an excellent question. You have to try and eliminate as many variables as possible. Here is my process...

The room: I need to understand the acoustic characteristics of my room so that I can develop an ability to listen “through” the room. If I am evaluating speakers, I need to spend quite a bit of time experimenting with positioning to try and get the best out of them. I have a listening room on the acoustically lively side. It helps to have another room that has more damping just to double check my conclusions. Although, with experience this becomes seldomly necessary.

Partnering equipment: I use a minimum of 3 DACs, 3 amplifiers and 3 speakers to evaluate a new component. I have owned these components for some time so instinctively recognise their sound characteristics. Although, I mainly listen through my main system if I am evaluating a new piece of equipment, it is important to try any new component with different combinations so that I can determine how it behaves and what might be suitable partnering equipment. For example, my Exposure Pre/Monos have a big, warm, rich sound where as my Hegel H160 is much leaner and more clinical in its presentation. My Audiolab M-PWR is a compact 40 watt amp that shows if a speaker is difficult to drive. Listening material: Having test tracks that I know inside out helps because I know instinctively how it should sound ordinarily and can more easily identify changes. Tracks are selected based on their ability to highlight a specific aspect of a products performance. For example, I have certain tracks that I listen to for evaluating female vocals, soundstage, imaging, transient response, etc. Most of them are great quality recordings but not every track because it is also import to know how forgiving a component may be of less than perfect recordings.

AB testing: I resist the temptation to switch components after minutes or hours of listening. Short term auditory memory has been shown to be very unreliable, even the order in which you switch components makes a difference. Have you noticed that when manufacturers do this they always start with the cheapest component and then go to the more expensive one and not the other way around. There is a very good reason for this. The second time you hear something new, it will sound better even if you haven’t changed anything. IMHO the only way to evaluate a component is to listen to it for days, making notes that you refer to and update through the process. Product burn-in may be a hotly contested debate but human burn-in is definitely an element to be considered when evaluating any new component. Give yourself time to adjust to how a product sounds, allow yourself to slowly form an impression, once you feel comfortable, only then, change a component. Is this process scientific? Absolutely not but I know of no better way to evaluate new components.

Thank you for the question. I may do a video on this topic."
I find that listening over the long term is a much better way for me to evaluate the differences that components make.  

That's not to say that doing A/B testing doesn't point out differences, it does.  But what it points out is that there is a difference, not so much what the difference is. 

Sometimes it's easy to define the differences you hear doing a quick A/B test, but for me at least, it takes extended listening to pick up on the more subtle nuances that a change in gear makes and what those nuances are.