The Audio Science Review (ASR) approach to reviewing wines.

People here get their pants in a bunch over ASR, its contributors and its methods. You would think they were doing something illicit. Their methods are explained and disclosed, their results are published, the results are open for comment. As that goes, there is nothing objectionable. I am not sure the ranking differences mean all than much as concerns SINAD numbers between closely-rated devices. Some of their speaker ratings, when done correctly, provide useful indicators as to which speakers can be expected to perform well in on and off-axis listening. They don't make useful comparisons of other features of a particular device aside from SINAD, Spinorama and frequency response. They do point out response irregularities and comment whether the deviations are likely to be audible. The commenters are a mix of thoughtful and knee-jerk, and you have to filter out the latter.  I don't agree with the wine analogy in the OP, but if you were to extend the wine analysis to things like acidity, sugar content and tannins content you might have a better comparison. Unlike other reviewer outlets, ASR isn't heavily supported by advertising like most of the audio print media. They have posted unfavorable reviews of generally successful if not popular products and have given good reviews for little known and inexpensive new products. What I think bothers readers here about ASR the most is their objective methods and general lack of favoritism, which forces those who disagree into the inherently weak position of attacking their methods.

This post gets at a real value for both A'gon and ASR -- objectivity:

Unlike other reviewer outlets, ASR isn't heavily supported by advertising like most of the audio print media. They have posted unfavorable reviews of generally successful if not popular products and have given good reviews for little known and inexpensive new products.

Avoiding marketing and forum hype is a good thing. ASR does it their way. Some do it on A'gon with their own experiences or counterarguments.

These two posts get at a problem with over-enthusiasm about "measurement" and the religious fervor of some at ASR:

I'm unhappy that they are most likely steering newer audiophiles down the measurement rabbit hole, when they don't even have a clue yet what type of speakers and equipment they personally enjoy. 

The decades of going into audio stores in my region and listening to a wide array of systems is something I cherish to this day. Going to listen to other peoples systems, all great learning experiences.

Both comments point to the fact that an overweening reliance on measurement tends to keep people from experience as critical to learning to listen. Trusting one's ability to take time to listen, notice, feel, and connect with music is the key to good audio, and when rankings and measurements displace that, we bind ourselves to a technocracy that shoves experience (and value) aside. This doesn't mean that measurement cannot help experience, only that it needs to be watched as carefully as an open flame.

chenry

... What I think bothers readers here about ASR the most is their objective methods ...

There is nothing inherently objective about measurements and the ASR site is dripping with bias and disdain. I don’t take ASR seriously and I’m not fooled by the use of "science" in its name.

the religious fervor of some at ASR

No such thing at Audiogon. Right?

Well I guess since Amir admits he listens to less than 50% of the products he reviews, those who believe a proper review can be conducted without listening, should be comfortable with ASR's methods. To me even the suggestion of a review based simply on measurements should cause even the faithful to run for the hills.