What should be mandatory in every professional published review-


When testing a company's newest amp, preamp, etc, and it is a refinement of a prior product that was on the market, ie, a Mark II, an SE version, a .2 etc, it should be mandatory that the review includes a direct comparison with the immediate predecessor. IMHO, it's not enough to know ion the product is good; it's also important to know if there is a meaningful difference with the immediate predecessor.

I'm  fan of Pass Labs, and I just looked at a review of an XP22 preamp. I find it very disturbing that there was no direct comparison between the XP22 and the XP20. And this lack of direct comparison is ubiquitous in hi-end published reviews, across all brands of gear tested. I don't blame the gear manufacturers, but rather the publications as I view this as an abdication of journalistic integrity.

 

Opinions welcome- 

zavato

would be nice, but I agree, completely impractical/impossible to always have the previous unit on hand for comparison...and then, there are so many other products it would be great to compare it to...not just the XP20 but an ARC, a CJ, JRDG etc. 

I would place the responsibility on the manufacturer to explain/demonstrate to their clients what is different between the former version and an upgraded or updated version.

I place the responsibility on the reviewer to,

  1. At a minimum, use partnering equipment that would reasonably be expected to be used with the review equipment - i.e., no mis-matches, and to
  2. Compare the reviewed equipment with comparable competitors available in the market at the time of the review.

I was fortunate to have seen reviews by the same reviewer, of my new DAC and of the same manufacturer’s former DAC. The direct comparisons were certainly helpful in knowing what I should expect. However, that will mostly not be practical, as several here have pointed out. 6moons usually does a good job of making meaningful comparisons, IMO.

What makes someone a professional reviewer?

We get paid to write reviews, we are willing to hump lotsa different gear in and out of our systems and spend several hours ferreting out all the details of a component and how it compares to something else (unless you write for TAS), and we’ve had the advantage of the perspective of hearing a lot of equipment in our own rooms and systems.  Plus, we can write well enough to both convey thoughts and concepts in a way that people actually like to read.  

That out of the way, personally if a manufacturer would go to the trouble/expense of shipping a current and former version of a review product I would’ve been happy to write about the differences (I actually did that once).  But, that’s rarely offered and would greatly increase the cost to the manufacturer with shipping costs and such.  But, and much more doable and what Soundstage.com always required, was that any reviewer had to at least have a comparable piece in their review system so there was at least one applicable point of reference.  Anyway, trying to do what you’re asking would fall mainly to the manufacturers and not the review publications unless a reviewer happened to have the previous iteration of a component, which happens but is relatively rare.  
 

If you have never visited Fredrik Lejonklou's website

it is a lesson in how to present a products evolution.

I wish all makers would follow suit.