What is the standard for judging a systems sound?


It is often said in these threads that this hobby is all about the music. That live music is the only meaningful standard for comparison when determining the quality of a stereo system. While these words sound good, are they really true?

A violin should sound like a violin, a flute should sound like a flute, and a guitar should sound like a guitar. Many purists will immediately say that amplified/electronic music cannot be used as a standard since a listener can never really know what the intention of the musician was when he/she recorded it, and what that sound should be.

Even something as simple as an electric guitar has multiple settings from which to choose. Electronic keyboards have hundreds of possible voices, so how does the poor audiophile know how the tone was supposed to sound?

These are valid concerns. Back to the purists!
“That’s why only unamplified classical music can be used as a standard!!!” On face value that looks like an acceptable statement. Consider some facts though. In my immediate family we a have several musicians who play a few different instruments. We have an electric piano (due to a distinct lack of room for a baby grand), acoustic guitar, Fender Stratocaster electric guitar, a nickel plated closed hole flute, a silver plated open hole flute, a viola, and a cello.

I have a fairly good idea how each of these instruments sound. One comment I must make immediately is that they sound a little different in different rooms. Another comment, which demands attention: when I bought my first flute I knew nothing about flutes. I began fooling around with it and enjoyed the sound. I liked it so much a bought a better, as mentioned silver open-hole flute. This flute sounded much better than the first flute. The tone was richer (the only words I can think of to describe the difference).

The reason for that background information is to show that the same instruments in different room’s sound different, AND different models of the same instrument have a much different sound!

If we audiophiles are using live unamplified music as a standard there are still several important issues, which must be addressed. How do we really know what we are hearing? What instrument is the musician playing? Was that a Gemeinhardt or Armstrong Flute. What are the sonic characteristics of the specific instrument. Stradivarius violins sound different than other violins, if they didn’t people would not be willing to pursue them so aggressively. Better instruments (theoretically anyway) sound better than lesser instruments. The point here is that different versions of the same instrument sound different.

I have seen the same music reproduced in different settings. I have heard string quartets play in a garden in Vienna. I have heard the Pipe Organ in Stephan’s Dom. I have heard Rock and Roll in arenas and Performing Arts Centers. I have heard jazz played in small one room clubs, not to mention the above listed instruments played in the house.

Each one of these venues sounds different from the other.

When I am listening to a selection of music at home, how do I know how it is supposed to sound? None of the LPs sounds like any of the particular places I have heard live music, while none of those places sounded like any other either.

There is no standard by which to judge the quality of live music since no two venues sound alike. If everyone were to go to the Royal Opera House in Covent Garden and hear Tchaikovsky’s Symphony No. 6 would everyone hear the same thing? Even if they did, and that one concert became the standard by which all other recorded music was judged, would that be translatable to allow the judging of all other music?

I have never heard a cello reproduced as well as my sons playing in the living room. I have never heard better flute players sound better than my own terrible playing at home.

So what do we audiophiles really use as the standard by which recorded music can be judged?
128x128nrchy
Pubul57, despite the rhetoric about the standard being unamplified live acoustic music - a standard that (within limits) is still useful and admirable, and which I understand aspiring to - I think that in actuality, most gear is essentially reviewed in relation to other gear the reviewer has experience with. Which makes perfect sense to me, since the different sounds of components will all be more similar to one another than they will be to real live music (a large part of the reason for which has to do with the recording process of the source material fed them, BTW).
Zaikesman you are absoluely right about reviewers, both here are in publications using other gear as the standard. I don't think anyone's reference system is a valid comparison by which to measure new equipment.

No one else knows what your, my, or John Atkinson's system sounds like in the appropriate listening room. If I tell you that my new Klyne pre-amp sounds much more like live music than my old Krell pre-amp what good does that do anyone? I could be completely right and still not be making a statement that does anyone else a bit of good. For other gear to be a valid reference point everyone has to be familiar with the gear being used as a standard. Then they need to be familiar with the cabling and room in which the equipment is heard. No two rooms sound alike.

Live music is not a static target, but neither is anyone's refernce system. Zaikesman, how long has your system been the same? We neurotic audiophiles are in a constant state of flux. My system went unchanged for about a year, in the last month I bought a new pre-amp, and I'm in the process of buying new speaker cable. Nothing that changes so much can be helpful to others who do not hear that system. In the case of reviewers their system changes from week to week. How can any of them be used as a refernce. They don't even know what their systems sound like. They just know the latest interation of that system.

Zaikesman, I don't know that we really disagree in the essence of things. We both want our systems to be as good as reasonably possible, it's just that we have trouble locating a standard. I'm sure your mastertapes don't sound the same at home as they do at the mixing board or as the event in the studio. If you could actually use mastertapes you would definately have a leg up on the rest of us, but where do we stand in relation to the cello, flute or piano I hear at home?
Nrchy, I think I agree with your point about the pitfalls in using reference
systems as a point of comparison. But if you follow this line of thinking
to its logical conclusion, then it means virtually all of the advice given
within these forums regarding the sound of components is worthless.
You haven't heard my system in my room and I haven't heard yours.
What valid conclusions about sound quality can we really draw from the
words others have typed into their PCs?
To partially answer both Nrchy's and Onhwy61's questions (How long has your system been the same? What valid conclusions can we draw from the observations of others?) as they regard myself, here's some news FWIW:

Yesterday I bought an amplifier from a local Agonner, paid cash and picked it up. I did this not because I necessarily wanted a new amplifier, but because I've been planning on doing some modifications to my current amps (capacitor upgrade) and wanted a spare. However, I do confess to having another, additional motivation. My regular amps are VTL MB-185 tubed monoblocks, but for a while I have been somewhat curious to see how I would react to a modestly-priced solid-state stereo amp - like the ones I used have a few years back before I got my previous C-J tubed stereo amp, only more up-to-date. Call it a reality-check.

The amp I got is a McCormack DNA-125. My reference amps cost about $5K new, the DNA-125 about $1.7K new. Truthfully, I am still only warming this thing up, as I literally inserted it into the system less than 12 hours ago. But already I can tell there are some things it does at least as well, if not better (or merely as well but differently) as my reference. Based on my past experiences, I was in no way expecting to be seriously tempted by this amp, and probably planned to sell it once my mods were done. However, I admit that part of the reason I'm auditioning something else now, before I do the mods to the VTL's, is to confirm in my own mind whether my reference should be a 'keeper' that I sink more money into, and also to double-check my overall commitment to tubed power by bringing in something solid-state. That the comparison could be called 'unfair' from a price standpoint (the output power isn't very different between the two into 4 ohms) I'll just have to let slide, since I wasn't going to drop much more than I did for a presumed temporary experiment.

But this little McCormack is obviously a very fine performer, maybe even 'good enough' - if we can ever admit to such a thing in this hobby - and I am going to enjoy listening to it for a while. Anyway, to get to the question touched upon by this thread, I just returned from rereading Jim Merod's review of this amp, and then going to audioreview.com, where there are 35 user reviews, who gave it an average of a perfect 5 out of 5 rating. And let me tell you, both Merod's piece and almost all of the user reviews are, I think at this early stage, dead-on acccurate in their assessment of this amp.

Now, this may not mean much in isolation, especially since it kind of surprises me, as I routinely find myself disagreeing with reviews both professional and amatuer. And, of course, I have a ways to go with this amp before coming to any final conclusions - there are still things I suspect my reference does better, although in terms of value for the money I likely have some thinking to do. But clearly, this McCormack is doing many things seemingly 'right', and everyone I've read about it so far thinks so too (next stop, Agon archives). That's got to mean something I believe ; that kind of consensus, jibing as it does with my own tentative experience, cannot be easily dissmissed as devoid of significance, at least not by me...
The last thing that I thought I would ever do is feel like I have to come to the defense of audio reviewers. The truth of the matter is that although audio reviewers are oftentimes easy targets for criticism/ridicule, wether we want to admit it or not, there are in fact a few truly talented ones. Have you guys read the work of people like Jon Nork, Harry Pearson and others, over the years? I don't know about you guys, but I have found their commentary extremely useful. They helped develop a language to describe what they hear that has changed the way that we all talk about the hobby. I frankly don't find it all that difficult to get a sense of what the best reviewers are trying to say about certain components, eventhough their systems are in constant state of flux, since they (the best ones) usually make reference to the sound of the real thing in a way that to me, makes sense. The same way that we need to attend a lot of live music performances to really be able to build a good personal reference, by familiarizing ourselves with the language and yes, biases of these guys, we can get some really useful info about components. I find the across-the-board cynicism directed at reviewers unjustified and counterproductive. Are there a lot of hacks out there? You bet! So what? Ignore them.

I disagree that all reviewers use other equipment as a reference. Yes, a comparison, even if not stated in the review, to other equipment is probably inevitable. So what? As long as the focus is on how it measures up to the sound of the real thing.

I'll say it again: There are many "generic" qualities in the sound of the real thing, that makes concentrating on issues such as the effect of the sound of different venues, mics, mixing boards etc., while certainly not irrelevant, almost pointless. We just need to familiarize ourselves with these sonic qualities enough. Perhaps spend less time and energy on feeding the neurosis of the hobby, and more time attending live events. Not an essential pursuit in order to enjoy music and the hobby, but certainly essential if we want to bother with comparisons using any kind of meaningful standard.