Why HiFi Gear Measurements Are Misleading (yes ASR talking to you…)

I have never heard a haze when listening to an SS amp,

Likewise.  I've done plenty of listening to SS and tube amps, never heard this SS "haze" of which he spoke.

Some people project their own impressions as some objective truth, as if just claiming something means it's true and "has to be explained."

Amir showing up here is like Richard Dawkins dropping in to a Church revival ;-)

 

 

 

 

@nyev

 

But over time, and not a heck of a lot of time, say 2-3 weeks of “living with” a component, I find that I can arrive at a more stable and true subjective assessment of a component. One that personally, I find goes beyond what the measurements can tell us.

 

The thing is, if you are starting with a flawed methodology - in your case sighted listening - using the same flawed methodology over a longer time doesn’t yield more reliable results. Biases can modify, or settle in over time, and you may be attributing this to the gear rather than changes in your perception, which has been Amir’s point.

The thing is most of us really don’t want to be told...or learn...our perception isn’t reliable. We stake so much on it. But, reality doesn’t bend to our desires, human foibles are what they are.

Now in talking about blind testing like this the hackles will often go up "hold on, so you are telling us we can’t rely at all on our perception? Now we have to blind test everything or we can’t have an opinion or do this hobby?"

No. Not exactly.

First of all, clearly our perception IS relatively reliable. We get through the day using our senses. And blind testing is not easy, and depending on the gear can be utterly impractical. I don’t think any audiophile HAS to engage in blind testing or pay attention to audio science. We can all do whatever we want.

HOWEVER, the fact is we have biases and they are relevant. If we don’t acknowledge that variable, we are simply being ignorant.

So given blind testing (and often, measuring) is impractical for many audiophiles, how to navigate these problems? One way is to just say "look, I don’t care about measurements or blind testing, I’ll go just on what I seem to hear." Fine and dandy for anyone who wants to do that. But IF you are going to take your impressions and make claims to other audiophiles based on those impressions, especially if they are dubious claims in light of generally accepted technical theory and practice, then you shouldn’t be surprised if some audiophiles hold a skeptical opinion and point out they would prefer more rigorous data - e.g. measurements/listening tests controlling for biases - before they accept such a claim about said piece of gear.

So nobody has to do blind testing to conclude what they want, but IF we want to be more careful about conclusions - learning from what the scientific method has to tell us - THEN it makes sense to look for evidence that is less susceptible to run of the mill sighted bias effects.

And, though blind testing may be inconvenient for most of us, there is plenty of engineering, testing, science out there ALREADY done by competent people, that we can look to when trying to evaluate a claim about equipment.

So, the way I navigate the problem of the impracticality of blind testing everything, vs recognizing the variable of human bias is basically a heuristic like ’Extraordinary Claims Require Extraordinary Evidence.’

So, if someone tells me they auditioned some Wilson Audio speakers and some MBL speakers and they describe the differences they heard, I’m perfectly happy to provisionally accept they heard those differences. Sure, could be some bias involved, but it’s also extremely likely they heard real sonic characteristics in each.

That there are audible differences between speakers is well established by theory, practice, experience, measurable evidence and even listening tests controlled for sighted biases.

That’s not the case for some of the claims in high end audio, though. The claim that an expensive USB or AC cable will likely alter the sound from any cheap (competently designed) cable IS quite technically controversial. Many knowledgeable people (the ones not selling such stuff) will explain why this his highly unlikely given how those things work. So...for THAT type of claim I personally will raise the bar for the evidence. I’ve had the personal experience of "hearing obvious sonic differences" between cheap AC and high end AC cables, but when I performed blind tests where I didn’t know which was being used, none of the sonic differences were there at all and my guesses were random.

So when an audiophile swears up and down he heard something "so obvious" when changing a USB or AC cable, I’ve learned that we really can have very strong but erroneous impressions (something science has told us for a long time).

It won’t matter if the anecdotes pile up, because they are all using the same method that allows for sighted bias. I’ll wait until I see measurements showing actual changes in the audio signal and/or people able to reliably pass blind tests for choosing between such cables.

Neither you, nor anyone else here, needs to have the same criteria I do.  Follow your bliss.   But there ARE good reasons for having such criteria and it shouldn’t be seen as some sort of heresy for which the skeptical person is made a villain, just by giving this reasoning.

 

Cheers.

 

 

 

 

 

@prof , like Amir’s points, you make rational points as well.  But really what we are debating is, which approach is more flawed - our senses because our biases get in the way, or measurements because while they reflect “reality”, they do not reflect on how we perceive a component’s physical performance.  For me, I believe the latter is far more flawed.  It doesn’t mean that the flaws you are pointing out in subject assessments are not valid - they totally are!  But IMO it’s the best we have, barring the day we can figure out  measure human perception of physical stimuli.  Once we can do that, we are into Bladerunner territory!

@amir_asr

mp3s are not great. Sure, you could fool someone in to thinking that 2 files are the same on a smartphone over bluetooth, but upon further inspection; in a more resolving system, you could tell the original .wav file and .mp3 file apart easily, no matter what the kbps was, even 320 kbps.

For example there is a feedback loop from the brain to the hearing system to seek out information in a noisy environment. This is the so called "cocktail party effect"

This effect does not exist in a quiet listening environment. You go to visit an audio shop. Walk up the stairs and they’ve got a listening room. No one else there but you and the sales guy.

You’re at home - in your office listening to headphones and/or speakers. The room is at probably 30 dB, perhaps even less. My office for example can be a bit lower than that. The music playing on speakers and headphones will overpower the environment in this case for 2 reasons -

First, the level is louder and...

Midrange frequencies that are louder or within the same octaves effectively cancel out other midrange frequencies. Bass and treble would also soften the resolve of background noise.

I’ve found a bunch of posts on other forums where people are refuting your measurements with their own. I will share them later today....just been busy.

 

 

This effect does not exist in a quiet listening environment. You go to visit an audio shop. Walk up the stairs and they’ve got a listening room. No one else there but you and the sales guy.

Yes this "effect" does happen in quiet room. The ability to pick out relevant details in a noisy environment is just one outcome of how the brain adapts. Our hearing including our ability to extract details, hear artifacts, etc. is not static. It is task dependent. At a lay level, it is called selective attention. At a neural level, our brain rapidly adapts neural weighting to the tasks on hand, which means if you are looking for discrepancies in how you think something should sound, you are far more likely to hear them as opposed to them just being "background" information.

The process whereby you adapt to a new piece of equipment is also related. Initially it is new, so you are looking for artifacts, differences, changes. If there are really changes, you are more likely to find them, because your brain has rewired to actively look for them. It will also find things that were always there that you never noticed. Over time, you/your brain settles, and you are back to listening to the music.

 

mp3s are not great. Sure, you could fool someone in to thinking that 2 files are the same on a smartphone over bluetooth, but upon further inspection; in a more resolving system, you could tell the original .wav file and .mp3 file apart easily, no matter what the kbps was, even 320 kbps.

How confident are you that if presented with only an MP3 file, 320kbps, that you could accurately state that it is MP3?