Audio Science Review = "The better the measurement, the better the sound" philosophy

Reliability and consistency are big issues that I think are overlooked by all reviews, somewhat out of practical necessity. To really do it right is quite expensive and time consuming. Multiple units should be tested over long periods of time with a variety of different inputs and outputs and configurations. My personal experience with digital audio and video equipment is that intermittent problems can and do pop up that are audible and visible without totally destroying the signal. The "bits is bits" argument clearly isn’t true. The music or video can come through distorted at times but mostly intact, meaning some bits are being misrepresented. After a system reset the problem usually resolves for the short term. If the effect is subtle it might be misattributed to a fundamental flaw in the quality of the media or in the playback potential of the component in question, when really it’s just an intermittent inconsistency caused by various devices not coordinating well with each other.

As for audibility of various cables and other electronic components, what might shine some new light on those and how they interact with each other would be a set of measurements off a speaker in an anechoic setting rather than test bench measurements of the components in isolation. It would be important to keep everything identical in the measurement space each time, only changing equipment upstream. I suspect that some unexpected differences in the speaker’s output might be measured as amplifiers, dacs, cables, etc. are interchanged. To keep things exactly the same with each measurement the speaker would probably have to be built into a wall so that it’s front side was in the anechoic measuring space and all other equipment was outside, preventing any potential effects from the size and shape of the components changing the response in the measuring area. Even that might be a bit misleading because some equipment might be effected by being exposed to the sound the speaker is making in a sort of feedback loop. So a second speaker might have to play in the equipment room to provide that input, but it would be very important that none of that sound leak back into the measuring area. You can see why this sort of testing isn’t done very often.

A scientifically established fact on limits of human perception can easily be misapplied. Take for instance the claim that a light blinking fast enough will appear to be not blinking to the human eye that’s looking straight at it. This is true and demonstrable, but someone might say something like "If it’s blinking at a certain speed you can’t detect with your eye that it’s blinking." That’s not true, because as you sweep your vision across you do detect that it is blinking. So there’s a fact that’s been well established and then there’s a story about that fact that isn’t really true. I don’t know that I can come up with a really good comparison with audio but one might be masking effects. MP3 data compression relies on known tonal masking effects in human perception to throw away data that you "can’t hear" anyway. This may work better in some situations than others. I recall listening to a recording of the cocktail party effect, where on the recording you can hear a whole bunch of voices talking at once but can’t really make out a word anyone is saying because they all mask each other. In stereo over speakers I still couldn’t make out a conversation. Because it was, I believe, a binaural recording using a dummy head, when listened to over headphones or using a divider plate between my speakers to prevent crosstalk, I suddenly could make out a nearby conversation because spacial cues became available to more distinctly separate it in distance from the surrounding hubbub. So why am I bringing this up? I’m just wanting to acknowledge that while I really appreciate reviews with measurements like ASR does, I do acknowledge that there’s a strong possibility that there are things being missed that are audible. I think it's a worthy effort to keep trying to find ways to measure everything we perceive and enjoy in audio. With more understanding we have a better chance of more consistently getting the perception that we want.

asctim

... while I really appreciate reviews with measurements like ASR does, I do acknowledge that there’s a strong possibility that there are things being missed that are audible ...

That's especially true because ASR doesn't bother to listen to everything it measures. Without correlation to what we hear, the measurements don't have much value.

Audio Science Review. Is interesting but measurements are just that. Several amps might have similar specs but one could sound bright, another less bright but bass strong, the next might have a great Soundstage but lack midrange. I heard the difference. I will never forget the Mark Levenson monos system I heard in the 1980s using a Lynn Sondeck, wow! I don’t remember the model. I think they said it was 10wph? Unbelievable. Really thumped low and sounded perfect to me.

ASR claims that all amps with the same frequency  response and output level matched and that  are load invariant will sound the same.

There probably is a degree of truth in that but  it ignores things like harmonic distortion types ,skew rate and resolution of stereo information.