@hilde45 I was thinking about this type of testing where there’s a group of people listening to different components, in this case, transports.
In my opinion, I would not rely on this type of evaluation:
1. The evaluation is done by a group of people not thoroughly and intimately familiar with the system.
2. It was a quick A/B type comparison.
Reason why I would not rely on this -
A listener needs time to evaluate a change brought by a component in a system they are familiar with. This takes hours, if not days, listening to one component then repeating the process with the old component back in its place. Asking a person to evaluate a change in the system they’re not familiar with is not doing anyone or the component any favors. The brain needs time to get a good baseline and in this case, that time isn’t allowed.
For this reason alone, the results of this evaluation are pure entertainment and cary very little, if any, real world value. You have a panel of listeners with too many variables they’re put up against to make a proper judgement call.
This of course is just my opinion and how I see and treat these types of listening tests. This is also why a blind A/B is bull 💩. It’s too quick and too flawed in its set up and execution.