Article: "Do Blind Listening Tests Work? My Sessions with the Colorado Audio Society"

Nothing wrong with them... they just are not attempts to recreate the real thing. 

This is a common standard. It assumes that what is "real" is:

• live music
• un-amplified (I assume)
• in some kind of venue (hall, church, club)
• from a certain distance
• in the center
• using certain microphones
• using certain mixing and mastering techniques

Many of the above factors involve interpretative choices. Are there "more and less real" microphones or placements of microphones? Are there more and less real mixing techniques? 

You see the point. Even in cases of live music reproduction, there are so many choices that the idea of a convergence on "real" begins to look like a hopeless quest.

What’s being sought is something interesting or stimulating or pleasing -- with perhaps the illusion that one is "there, at the concert." But even that goal can be accomplished in so many ways, that one cannot converge on a single (objective) solution.(After all, the person at the concert in the distant balcony, with lots of reverberation and an unbalanced frequency mixture is having a "real" experience at the concert. But is it less "real" than 7th row center?) As you said, there are many paths, even to the standard which you take as "real."

And if that’s true, then there’s nothing more real about live music than about electronic, mixed, multi-track, or live-with-reinforced sound (PA’s) music. The goal is always some kind of experience in the listener, whether it’s about the experience of "being there" or some other experience. 

Get rid of the word "real" and a lot of these issues just melt away.

There exist a "real" sum  of acoustics objective information any stereo playback system loose by definition of being a stereo system...(information about timbre and spatial localizations and attribute of sounds as holography,volume,listener envelopment versus sound source width etc)

When people speaks about recreating the real objective collective event of a concert, which is impossible for many reasons , what in fact is possible and what they refer to is the retrieval of the original acoustics information coming from the recordings  many trade-off "designed" and picked choices kept by the recording engineer but lost in the regular stereo "translation" process of these  acoustics parameters values in our own stereo playback system/room because of the cross-talk between two sources of sound (the speakers)  instead of one localized  source of sound as in real life  if i hear a bird for example or in the concert case.

The only one explaining this very clearly is the acoustician Edgar Choueiri in his many articles...

Then we must not get rid of the real but understood what real means from the  acoustics concepts and parameters at play in  the recording process to the playback process in our room.

Get rid of the word "real" and a lot of these issues just melt away.

Unless double blind testing includes long-term listening, my take is that while it can be useful to determine IF there is a difference, it is not useful in determining which is "better" to us.

Analogy: many times I have eaten at a restaurant and the first few bites of something taste very good. Then I realize it's actually very salty or has some other overly emphasized characteristic that doesn't withstand repetition in taste. It doesn't taste as good on the 5th or whatever bite.

I have found audio to be similar. What sounds detailed or something else at first can be fatiguing. 

So it's a useful tool, but far from the last word.

I agree with the point that all stereo playback systems inherently lose a significant amount of objective acoustic information present in a "real" sound event. (This lost information includes details about timbre, spatial localization, sound holography, volume, and listener envelopment—elements that define the characteristics of a sound source.)

I also agree that it is best to talk about "real" in connection with the retrieval of original acoustic information from recordings. 

What I think is misleading (carefully avoided by mahgister's post) is the constant reference to the original, live musical performance event as the standard of what "real" means. As was pointed out, this is impossible to capture and so is a misleading standard. That’s why I would prefer to get away from the world "real."

FWIW, the term "fidelity" (as in "high fidelity") is a better word, because one can be faithful (have fidelity) without being bound to perfectly reproducing an event which, in its totality, is gone forever.

Arriving at one's reference system should probably take many years of trial and error, listening to live music of all kinds, listening to a wide variety of systems in a wide variety of settings/listening rooms. I too often focused on a particular area of presentation, for instance at one time I became totally enamored with mid range timbre to the detriment of pretty much every thing else. I set up a system with that  'golden glow' tube sound, god how I loved the romance of that system. But over time I began to notice I was pretty much only listening to acoustic folk, anything with rhythm, energy, denseness was rather incoherent. Also overly resonant, one note type bass, highs rolled, this all became too apparent in time. So next I go in completely the opposite direction, soon enough analytical presentation becomes unsustainable. So, then the goal becomes a happy mean.

And so I always knew how 'real' music should sound via attending concerts, singling in choirs, choruses, bands. Why I went off in these two opposite directions remains a puzzle, just a part of the learning process I suppose. The other thing that greatly impinges on the ability of many of us to reproduce our 'reference' sound is the funds required to reach this goal, no doubt lack of 'reference' level equipment had a great bearing on my audio travails.

So now the one thing I've begun to notice over these past few years is how my choices in listening material has changed. I've found myself listening to far more acoustic instrument recordings, way more 50's into early 60's jazz, pop, instrumental, baroque classical, folk music than I previously listened to. In making this rather unconscious choice I've come to understand I now prefer listening to music that presents a more 'natural', less processed sound. Since I now have a far better balanced and/or 'natural' audio setup I desire to hear that same 'natural' acoustic in the music I play. Amplified and synthetic instruments and/or highly processed recordings don't have this 'natural' presentation or reference to live music. Hearing this more 'lifelike' presentation from both the recordings and system is engrossing to the point I simply don't get around to playing the 'other' stuff. The other salient point with the recordings I most enjoy listening to, is vast majority are from the era when tube equipment dominated in the studio, I can easily discern the early solid state recordings from the tube which started mid 60's. One other thing is how much more natural mono recordings have become in recent years, far more spacious sound stages vs when I had lesser equipment. Stereo very often sounds more processed in the sense it presents a tailored vs natural sound stage. 

And so in thinking about how all this relates to double blind testing. I'd suggest I'd have great difficulty in arriving at valid conclusions listening to a system I have very little exposure to. Certainly its possible I'd be able to differentiate between the component or components under study, but how does that hold any validity in the case I'm determining whether I want to purchase said component for my system. And then there is the question of sympathetic matches, this not some objective system this component being placed in. So, for me I don't understand the whole point of blind testing for audio, sure it can exercise one's analytical skills, but I'd suggest going to wide variety of live musical events will go much further in developing those skills.

Every time there's a discussion about double blind testing I go back to my early days in audio. Many establishments had  these switching boards where they could switch out components on the fly. The thing I most recall was being stressed out by trying to prove I had good listening skills, performance anxiety big time. Only recall inner confusion, this in spite of what I might have been saying. Never purchased a single item using this method, in home demo's were my primary way of purchasing back in the day. So my question is, why is present day blind testing still existent? Sales or some nebulous argument for improving analytical listening skills?