Reviewing the Reviewers


Check out http://www.high-endaudio.com/index_ac.html and follow go to the "Audio Critique" page, and then to "Reviewing the Reviewers" page.

This site is run by a man named Arthur Salvatore. He has written much about all aspects of audio on his site...his recommended components, his recommended recordings, his store, etc. He writes like a lawyer, but it seems like he actually has integrity...he must not be a lawyer. :-) Seriously...anyone interested in a point by point analysis of modern audio reviews should check out this site. He's analyzed many reviews and developed his own list of "rules" that most reviews tend to follow (and he's dead-on)...usually because the writer doesn't want to say anything negative about any particular sponsor's (or buddy's) product.

He received an angry letter from Michael Fremer. The letter and his analysis are included on the site. It makes for a long read, but it can be fascinating. Besides...it's information than every audio joe (or jane) should be aware of when they read any review...especially when they're planning on pruchasing a product highlighted by a particular review.

If you want to see textbook examples of his "rules" put into practice, just check out any Soundstage review written by Marc Mickelson.

Enjoy...
phild
Slawney, would you really say that Salvatore's critique of "these institutions" you mention is a "systematic" one, as you call it ? (Unless of course by systematic you mean a critique of the alleged "systems" behind those institutions and not the critique itself being systematic in its methodology and purpose) Besides that, I find your thoughts interesting and indeed an inquiry along the lines you draw out for us, could show us publishing policy, but hardly, I venture to say, something about the spirit and the integrity or lack thereof, of the individual journalist, who sometimes may be lead by a large rope and sometimes not.
Detlof, the analysis I am suggesting, I believe, is a reliable method of making use of audio magazines to inquire into the spiritual (intellectual) currents of an institution (in this case, a long-established audio magazine), not of the intergrity of an individual journalist, as you say, except to the extent that that journalist embodies the systematic interests and principles of the institution for which he works. Such journalists are not unknown. I think we all know a few. In an age (and a country) in which both economic and intellectual production has been so thoroughly rationalized (as in America), it is the norm that audio magazines like Stereophile and the Absolute Sound work fairly systematically in comparison to underground publications. One sign of this is the way a publisher approaches individual reviewers with specific projects and sometimes very specific plans. Readers are making a great mistake when they see reviewers as mere gatekeepers or yes/no sayers, instead of as experts in a publishing policy who are intelligent enough to follow through on this policy rather systematically by drawing up plans (with or without their editors) for their articles before writing them. The audio press reading public is also wrong (as you say in your admirable post above) to set their idealism against a publisher's materialism, instead of treating their own ideas in such a way that the publisher will be tied to them for the sake of their material interests. What if a large number of audio enthusiasts started to think otherwise about audio technology than Stereophile and the Absolute Sound? I think that these magazines would be desperate to find out how their readers' ideas have changed to reestablish their customer base and market share. As far as Salvatore's critique is concerned, it is systematic to the extent that he outlines some of the principles and policies that reviewers obey when they write and he has even drawn up a chart showing the increasing number of advertisements in Stereophile and the increasing number of components on its "Recommended Components" list. This is an important aspect of a systematic critique which would also have to make use of sociological and financial data that Salvatore does not possess in order to further lay bare publishing policies. Such a critique would probably expose the conception of the audio magazine as an organized patronage/lottery system that I mention above as a common annd erroneous conception of publishing. Who should do this critique? It would probably be a perilous and thankless task. Look at some of the inflammatory reactions to Salvatore's website (not only the responses from journalists, but also from audiogon members). Perhaps it was even wrong of me in this thread to point to the political theological dimension of Salvatore's project (his similarity to Martin Luther) since such comparisons are easily parodied to his disadvantage. I still feel it would be illuminating to call for case-by-case opinions from audiogon members about Salvatore's own "Recommended Components" and "Supreme LP Recordings" and his own sense of fair play (the dispute between him and Fremer lies in Salvatore's site's negative assessment of the Rockport turntable and the Stereophile recommendation of it as an A+ analog component of the year, to a large extent) since Salvatore cannot be approached completely only on the basis of his 'Reviewing the Reviewers' page alone. Some audiogon members have already given a few case-by-case opinions about Salvatore's other pages (one member already indicated that Salvatore's recommendation of a Parasound CD player shows his analog bias, and so on). I would like to see more case-by-case opinions of Salvatore rather than general put downs. Personally, whenever I find I disagree with Salvatore on some component or LP recording, I cannot help but feel that he should nevertheless excercise more of a decisive influence on readers (especially young readers) than some of the big audio magazines.
Slawney, thankyou for your thoughtful response to my query. To my mind at least it should be difficult to refute the points you make in your brilliant analysis. The methodology you suggest, is, as you say, difficult to apply in practical terms, but it points a way to help us hone our critical minds, helps us to question rationally and "systematically". I very much agree with the conclusions you draw in both your posts and wish to thank you for the time and effort you took to share your thoughts with us! Regards,
Slawney, "excercis(ing) influence" requires acceptance on the receiving end, IMO. i.e., it takes two to tango... one giving advice or expressing opinion, and another to listen, acknowledge, "intelligently". I think this isn't always the case.
It seems to me we have a dichotomy in the reactions to opinion (in form of reviews or other) even among seasoned audiophiles sometimes.
1) "Rational" reaction, that takes a stated opinion and tries to understand the parametres upon which the stated opinion is based, and accepts, rejects, or simply acknowledges a subjective experience ("opinion") stated by another person.
2) Emotional reaction, where the receiving end *indentifies self* with equipment or person reviewing equipment. We've seen posts that could be paraphrased as "what do you mean, *MY* amplification of choice is no good, those (MY) speakers are terrible???" Likewise, with one reviewer's credibility vs. another's, worse still (IMO), for newcomers who would give credence to a well published and thereby known reviewer and disregard *USERS* opinion. In this case, disillusionment seems unavoidable.
More so, since common sense easily evidences the commercial limitations imposed upon and followed by mags. In order to pay but lip service to the mags' obvious profit pursuits, there *must* be an emotional side that shroudes reality.

If not, then what is it?
Greg, remember the old Latin phrase "degustibus non est disputandum " ? One man's heaven (speaker, cable, what have you) is another man's hell. So even the most rational of statements in our field here, will most probably be met with an emotional response. The so called objectivists, who in fact are the most subjective amongst us here, try to cheat themselves (and us for that matter) out of this dichotomy, by maintaining, that most of what we hear is but a figment of our imagination anyway. To my mind, if we refuse to fall into that trap, we must learn to live with the dichotomy you mention, which in fact will become less and less difficult to bear, if we learn to accept, that most of us here will voice and shape their systems according to what their ears will tell them is "best", or at least approaching it and I suppose that there are as many "bests" out there as there are audiophiles around, who at this point in time happen to be happy with their systems. I've noticed, that the most seasoned amongst us will tell us about their systems and describe its musicality but will only get emotional if derided or hard pressed. They still tend to experiment and occasionally upgrade, but I would venture the guess, that they stick to their stuff longer than a relative newcomer with means. Also I've never heard them say that this or that gear is "best". If you haven't had the time to mature in the fairyland of audiophilia however, you tend to look for what is "best" and do so with much emotion involved. Its the mags, which tend to feed and accelerate this, because they have to find a new list of "bests" everytime they bring out a new issue. Nobody would read them, if they would not do so. They would bore us all stiff.
At the same time, they tell us, that when all is said and done, even the "best of best" will never approach the "absolute sound of real music in real space and time".
In peddling "best" in every issue, hinting that this month's "best" is most probably just a step closer to the real thing, than last month's, but at the same time maintaining, that the real best can only be real live music, they feed us what is called a double bind in psychology, which at best,( pun intended) will emotionalize us , at worst, drive us nuts and make us bust budgets in the process. It is also the mother of addiction. It is in this double bind, where to my mind the actual poison lies hidden, not in our suspicion, that the mags could possibly be corrupt. How to avoid it? Well, perhaps we should not entirely, because it nourishes the fire, underlying our quest for beauty. But too big a dose of it will make us restless, disatisfied, never content how "it sounds", because there might be something "better" just around the corner. My personal solution to this problem has evolved but slowly through the years: I've learnt neither to trust nor to mistrust the statements reviewers make, because I don't see any objectivity in them anyway, neither will I in mine or in anybody else's in fact. I've learnt, that there is neither a "best" in systems, nor an absolute truth in describing them. I've learnt, that I have my own tastes as far as musical software as well as hardware is concerned. In the course of time I've built a system just following my own ears and trying to voice and shape it in the way I wanted it to sound. To me its "best" emotionally, rationally I know that there are better around. I will read and listen to keep informed, I will experiment, as I do with the Bybee stuff right now, but there is a hardware foundation to my rig, which is neither the latest, nor the most expensive, which I know I will never change, because it comes closest to what in my mind and soul I find is musical. Newcomers to our field, I find, should be told of those basic rules which exist to make a system sound better.
They should be sent to as many live musical events as possible, to have a tertium comparationis between the sound of their system and the ideas of sound they have in their minds about how a system should make music. They should read the mags, but with the attitude of a sportscar enthusiast, who knows that reading about how a Porsche handles, will reflect the journalists experience, but not the feeling he himself will have, when he has his behind in the driver's seat, but above all he should learn to understand, that there is no "best", nor ever will be. If that is achieved, you're out of this double bind I mentioned before and if reviewers are music lovers of independent mind and soul or capitalism's slaves becomes a point of little or no consequence and the dichotomy wich Greg has pointed out so well, is just the "both sides of the same coin". Speaking of coins: just my two cents, sorry I was so long. Cheers to all,