What is the standard for judging a systems sound?


It is often said in these threads that this hobby is all about the music. That live music is the only meaningful standard for comparison when determining the quality of a stereo system. While these words sound good, are they really true?

A violin should sound like a violin, a flute should sound like a flute, and a guitar should sound like a guitar. Many purists will immediately say that amplified/electronic music cannot be used as a standard since a listener can never really know what the intention of the musician was when he/she recorded it, and what that sound should be.

Even something as simple as an electric guitar has multiple settings from which to choose. Electronic keyboards have hundreds of possible voices, so how does the poor audiophile know how the tone was supposed to sound?

These are valid concerns. Back to the purists!
“That’s why only unamplified classical music can be used as a standard!!!” On face value that looks like an acceptable statement. Consider some facts though. In my immediate family we a have several musicians who play a few different instruments. We have an electric piano (due to a distinct lack of room for a baby grand), acoustic guitar, Fender Stratocaster electric guitar, a nickel plated closed hole flute, a silver plated open hole flute, a viola, and a cello.

I have a fairly good idea how each of these instruments sound. One comment I must make immediately is that they sound a little different in different rooms. Another comment, which demands attention: when I bought my first flute I knew nothing about flutes. I began fooling around with it and enjoyed the sound. I liked it so much a bought a better, as mentioned silver open-hole flute. This flute sounded much better than the first flute. The tone was richer (the only words I can think of to describe the difference).

The reason for that background information is to show that the same instruments in different room’s sound different, AND different models of the same instrument have a much different sound!

If we audiophiles are using live unamplified music as a standard there are still several important issues, which must be addressed. How do we really know what we are hearing? What instrument is the musician playing? Was that a Gemeinhardt or Armstrong Flute. What are the sonic characteristics of the specific instrument. Stradivarius violins sound different than other violins, if they didn’t people would not be willing to pursue them so aggressively. Better instruments (theoretically anyway) sound better than lesser instruments. The point here is that different versions of the same instrument sound different.

I have seen the same music reproduced in different settings. I have heard string quartets play in a garden in Vienna. I have heard the Pipe Organ in Stephan’s Dom. I have heard Rock and Roll in arenas and Performing Arts Centers. I have heard jazz played in small one room clubs, not to mention the above listed instruments played in the house.

Each one of these venues sounds different from the other.

When I am listening to a selection of music at home, how do I know how it is supposed to sound? None of the LPs sounds like any of the particular places I have heard live music, while none of those places sounded like any other either.

There is no standard by which to judge the quality of live music since no two venues sound alike. If everyone were to go to the Royal Opera House in Covent Garden and hear Tchaikovsky’s Symphony No. 6 would everyone hear the same thing? Even if they did, and that one concert became the standard by which all other recorded music was judged, would that be translatable to allow the judging of all other music?

I have never heard a cello reproduced as well as my sons playing in the living room. I have never heard better flute players sound better than my own terrible playing at home.

So what do we audiophiles really use as the standard by which recorded music can be judged?
128x128nrchy
Nrchy, I think I agree with your point about the pitfalls in using reference
systems as a point of comparison. But if you follow this line of thinking
to its logical conclusion, then it means virtually all of the advice given
within these forums regarding the sound of components is worthless.
You haven't heard my system in my room and I haven't heard yours.
What valid conclusions about sound quality can we really draw from the
words others have typed into their PCs?
To partially answer both Nrchy's and Onhwy61's questions (How long has your system been the same? What valid conclusions can we draw from the observations of others?) as they regard myself, here's some news FWIW:

Yesterday I bought an amplifier from a local Agonner, paid cash and picked it up. I did this not because I necessarily wanted a new amplifier, but because I've been planning on doing some modifications to my current amps (capacitor upgrade) and wanted a spare. However, I do confess to having another, additional motivation. My regular amps are VTL MB-185 tubed monoblocks, but for a while I have been somewhat curious to see how I would react to a modestly-priced solid-state stereo amp - like the ones I used have a few years back before I got my previous C-J tubed stereo amp, only more up-to-date. Call it a reality-check.

The amp I got is a McCormack DNA-125. My reference amps cost about $5K new, the DNA-125 about $1.7K new. Truthfully, I am still only warming this thing up, as I literally inserted it into the system less than 12 hours ago. But already I can tell there are some things it does at least as well, if not better (or merely as well but differently) as my reference. Based on my past experiences, I was in no way expecting to be seriously tempted by this amp, and probably planned to sell it once my mods were done. However, I admit that part of the reason I'm auditioning something else now, before I do the mods to the VTL's, is to confirm in my own mind whether my reference should be a 'keeper' that I sink more money into, and also to double-check my overall commitment to tubed power by bringing in something solid-state. That the comparison could be called 'unfair' from a price standpoint (the output power isn't very different between the two into 4 ohms) I'll just have to let slide, since I wasn't going to drop much more than I did for a presumed temporary experiment.

But this little McCormack is obviously a very fine performer, maybe even 'good enough' - if we can ever admit to such a thing in this hobby - and I am going to enjoy listening to it for a while. Anyway, to get to the question touched upon by this thread, I just returned from rereading Jim Merod's review of this amp, and then going to audioreview.com, where there are 35 user reviews, who gave it an average of a perfect 5 out of 5 rating. And let me tell you, both Merod's piece and almost all of the user reviews are, I think at this early stage, dead-on acccurate in their assessment of this amp.

Now, this may not mean much in isolation, especially since it kind of surprises me, as I routinely find myself disagreeing with reviews both professional and amatuer. And, of course, I have a ways to go with this amp before coming to any final conclusions - there are still things I suspect my reference does better, although in terms of value for the money I likely have some thinking to do. But clearly, this McCormack is doing many things seemingly 'right', and everyone I've read about it so far thinks so too (next stop, Agon archives). That's got to mean something I believe ; that kind of consensus, jibing as it does with my own tentative experience, cannot be easily dissmissed as devoid of significance, at least not by me...
The last thing that I thought I would ever do is feel like I have to come to the defense of audio reviewers. The truth of the matter is that although audio reviewers are oftentimes easy targets for criticism/ridicule, wether we want to admit it or not, there are in fact a few truly talented ones. Have you guys read the work of people like Jon Nork, Harry Pearson and others, over the years? I don't know about you guys, but I have found their commentary extremely useful. They helped develop a language to describe what they hear that has changed the way that we all talk about the hobby. I frankly don't find it all that difficult to get a sense of what the best reviewers are trying to say about certain components, eventhough their systems are in constant state of flux, since they (the best ones) usually make reference to the sound of the real thing in a way that to me, makes sense. The same way that we need to attend a lot of live music performances to really be able to build a good personal reference, by familiarizing ourselves with the language and yes, biases of these guys, we can get some really useful info about components. I find the across-the-board cynicism directed at reviewers unjustified and counterproductive. Are there a lot of hacks out there? You bet! So what? Ignore them.

I disagree that all reviewers use other equipment as a reference. Yes, a comparison, even if not stated in the review, to other equipment is probably inevitable. So what? As long as the focus is on how it measures up to the sound of the real thing.

I'll say it again: There are many "generic" qualities in the sound of the real thing, that makes concentrating on issues such as the effect of the sound of different venues, mics, mixing boards etc., while certainly not irrelevant, almost pointless. We just need to familiarize ourselves with these sonic qualities enough. Perhaps spend less time and energy on feeding the neurosis of the hobby, and more time attending live events. Not an essential pursuit in order to enjoy music and the hobby, but certainly essential if we want to bother with comparisons using any kind of meaningful standard.
It seems to me that the recording itself sets the absolute limit by which we can evaluate the reproduction capability of our equipment. As has been said before, our equipment cannot transcend the recording itself, reaching beyond to a "more true" or "real" version of the actual musical perfomance recorded by the engineer and the processed into CD, Vinyl, etc.

I'm not sure why the "Objectivist" approach to evalauting equipment would not be the prefered approach to judging a piece of equipment's ability as a reproducer of a muscial signal. The less damage (distortion?) between source and "reproduction" (electronically and acoustically) the "better".

Now it may be that there are some aspects of music reproduction that we are not able to objectively measure, and some aspects that we do not know we should measure that are relevant to sound quality (psychoacoustically). I suspect that there many measurables that relate to the accuaracy of reproduction and that are psychoacoustically relevant.

If true, all we can ask a designer to do is to engineer equipment that "measures right" (reproduces the source with minimal damage). If they do that, we can say the equipment is good to the extent is accuratly reproduces the source and passes that information to the next string on the audio chain.

With equipment that can accomplish that, our critique of the "sound" coming from our stereo might be better directed towards the recording engineers that produce sound that is accurate to the orginal perfomance.

Whether this "accurate" sound is pleasant or unpleasant is up to the listener. It would not suprise me if many people prefer sound that in some way is a distortion from the original or add artifacts that were not part of the sound of original performance in real space.
Zaikesman, consensus does have value, as do the opinions of everyone who reviews a piece of gear, as long as it is done with integrity. If you tell me the Mc Cormack amp sounds good I believe it does. Based on your comments I would tell someone else that the DNA amp you mentioned is considered to be a good amp. The issue is what have I really learned. I consider you to be a credible source for information, but ultimately what has your comment done for me?

This is certainly not to say your comments are worthless, but it does call into question the value of your comments. I’m not trying to be rude, I’m just looking at the way we think about our systems. Is your determination of ‘good enough’ going to agree with mine, or anyone else’s?

Each instrument has a specific sound. That instrument will have a specific, but different sound when it arrives at the soundboard. It will have a slightly altered sound again as it is put onto a CD or LP. Which version of that sound are we willing to accept as being good enough, or as a standard which any or all of us can accept for judging the quality of our system? Or is it even worth it to try to have a standard?

The whole issue from the outset is my own wondering whether there was a standard that everyone can use to judge quality of gear. Other than anyone’s opinion it doesn’t seem that there is a standard. I believe everyone (almost) who’s posts and ideas I am familiar with on AudiogoN when they say something sounds good. I’m sure they think it sounds good, so in their mind it does sound good. Some people are less likely to have enough credibility to have their statements accepted by me as being reliable. But none of these comments are based on an absolute standard.