I have spent many years conducting research to understand how people perceive products of various kinds. The aim of this research has been to provide designers and engineers with feedback regarding how particular product features influence consumers' perception.
Differences in source material may well be factors in the perception of audio system quality. The way to resolve that is with a standardized set of source material that spans the range of potentially relevant audio system qualities. Whether you personally like the source music (or any other acoustic material) is not the issue. Professional taste testers face the same issue. However, they are trained to focus on the physical qualities of products rather than their own personal likes or dislikes. Musicianship might be an important quality of audio reproduction, but, if so, it can simply be included as a factor in the standardized sample of source material. In fact, a formal study to test this would be interesting to do.