The eternal quandary

Jimjoyce, when I evaluate new components I often take along recordings that are flawed in some way. For example, the system should expose an overly sharp guitar or lean sounding vocal. Its a good tool to see if the boundaries are defined. I did that with all the equipment I own.
I suppose at home when you I am being an 'audiophile' I really tend to select great recordings to show what the system can do - the optimistic approach. If listening to music, I select great music despite the recording and you have to be mentally disciplined to listen without critical ears, or the experience is ruined. i.e. let yourself be carried away with the art.
Twoleftears, agreed. The modes are very blurred, and its easy for to become critical again. I suppose if you keep reverting to being critical, you are bound to end up upgrading.

But I guess the question is: How do you know that the sharpness of the guitar or the leanness of the vocal is the fault of the recording, rather than the fault of the equipment?

I suppose one can make the argument that if some recordings sound good on a particular system, while others do not, then the fault must be with the recordings.

But then again, perhaps the problem is with the system, and a better system would not only resolve the problems with the bad-sounding recording, but also make the good-sounding recording even better.

I believe that most of us who have a deep financial/emotional investment in our systems would rather assign fault to the recording, rather than admit the possibility that the problem is with the system.

Perhaps a "sharp-sounding" guitar comes about because a recording engineer has "pushed the limits" in order to try get veracity on the recording, but this will be revealed on playback only by components designed in a certain way, and not on components that sound great with certain recording techniques.

Perhaps, in thinking we are overlooking the inadequacies of the recording in order to enjoy the music, we are actually overlooking the inadequacies of our systems?

I have plenty of cds whose "problems" (or what I thought were problems) have been cleared up as my system has improved. This has made me less apt to question the recording, and more apt to question my system. So, I think the question is valid: When we assign fault to the recording, how do we really know?

Jimjoyce, i think you learn which it is by experience. For example, I have a recording that sounds overly lean in most aspects, particularly vocals, but is otherwise very clean and detailed.
When i was shopping for my last system, i found out that some systems made it sound richer and more natural, and you could be fooled into thinking its an improvement. Then you try it on 3 or 4 reference systems, and you see it is a faulty recording. And so i use it as a boundary recording to sift out components that are too warm and rich. If that recording sounds good, i try a recording which i would say is overly rich and warm, and you virtually cant listen to it. Another example is a recording of a sharp steely guitar, and if it doesnt sound that way, you can rest assured the system will lack sparkle and life on a lot of material. I have many recordings that show up these types of things, and some hi fi stores look at me strangely when i use them, but they are very useful....of course if its your home system that you selected with great care over years, you dont want to discover any nasties....
So i suppose bias can creep in, but so what?
I like to think i am happy with my system, but there is always that next upgrade and tweak lurking. Shopping for hi fi can rightly make you highly critical, but at some point you have to really try listen to the music and write off bad recordings as just that, irrespective of the facts. If too many of your recordings have a problem, especially ones that are widely recognised as reference audiophile material by hifi reviewers, you probably need to upgrade.
By the way, I am sure i am not alone in reading hifi reviews with a view to trying the music that was used to test, rather than buying the equipment used.

John Atkinson and others have demonstrated that a high percentage of "popular" recordings issued nowadays are highly compressed. This is the work of the mixing/mastering engineer, and this is how that person wanted them to sound and to be released. But they're not very good sounding on "our" equipment, compared to the boomboxes and car radios that they were designed to be played on.

There's an ontological argument to be made here too. Most (though not all) pop and rock discs only exist in one version. They are what they are (rose is a rose is a rose), whatever the sound.

Most classical recordings offer different versions of the same composition. It think it was Gordon Holt who came up with the annoying and paradoxical "principle" (Holt's law) that the better the performance the worse the sound, and the related if subsidiary principle of the better the sound, the more trivial the composition being recorded.

There are other factors involved here. The early years of digital recording are known for "digititis"--that terrible, mid-treble glare that makes solo violin unlistenable and your ears bleed. Was that intentional? Did the recording engineer care? What were they hearing on their studio monitors? The inability to listen to large swathes of recent classical recordings is what got me,somewhat reluctantly, into SETs, which made the unlistenable listenable again.

Where is the "accuracy" in all this? What are the norms of accuracy? I certainly agree with Mike that if a disc sounds bad on three or four different systems, it's surely the disc's fault.

Actually I have found a more ( I suppose ) simple approach in dealing with the above mentioned problems:

I have LPs in my collection which I have used in a benchmark role through all the many systems I have owned in the last decades. I have- if you will- built my system around them. Later of course digital has also come into play. If my system meets those standards - and they have to do with timbre, prat, rendition of transients, soundstage, besides I'm particularly finicky with grand piano and female voices, as well as violins and big symphony orchestras, I relax. If it doesn't sound right, it is the take, not the system. Right or wrong, it helps peace of mind, as long as the benchmark takes - there are many - sound right. And those takes, being benchmarks to my ears come as close as possible to what I perceive as the real thing reproduced through my system.