Audio Science Review = "The better the measurement, the better the sound" philosophy

Here is the thing. A while ago I was looking around for a new DAC. I listened to about 8 and in no case was the cost of the DAC mentioned. I just sat in a chair and listened to the same music. One that I disregarded very quickly was the Topping.

This isn't a blind test. Not knowing prices isn't the same as not knowing which DAC you're listening to. Who controlled the payback? Third party who level marched ? These are some of the  reasons  I suspect your anecdote was dismissed on ASR . 

@cd318

Dare I suggest it’s a simple matter of vested financial interests?

The subjectivists approach actively encourages consumers to spend, spend, spend ever increasingly large amounts of money on equipment and cabling.

Most of them usually include an ad hoc performance league masquerading as a buyer’s guide that always correlates with increasing prices.

Which reminds me of Robert Harley’s contention, not just recently but also from some years ago, that audio components follow his own law of increasing/accelerating marginal returns (as opposed to decreasing).

So, if you spend another 10%, you will be gifted with something that is more than 10% better in every respect (appreciating that this has nothing to do with the level of expenditure, rather, a comparison between two levels).

It is odd how the forks at ASR and Amir's Army of Flying Monkeys clamor for blind test yet none are done at ASR.

These flying monkey's also call Amir a scientist because he has a oscilloscope, voltmeter and REW. I own a chainsaw doesn't make me a lumberjack. 

juanmanuelfangioii

It is odd how the forks at ASR ... clamor for blind test yet none are done at ASR.

Amir & Co. believe audio equipment can be evaluated without listening, so sometimes they don't bother to listen at all - blind or otherwise.

Those who clamor most loudly for blind testing rarely undertake such evaluations themselves.

@jerryg123 

@crymeanaudioriver and yet ASR is not doing any blind tests.

There are very frequent blind tests posted by members.  Here is a very recent one: 

His conclusion: "TLDR; NO DIFFERENCE could be reliably discerned via blind abx testing between the Hegel h390 internal DAC and the Chord Hugo TT2 under close listening in my listening room."

The problem with you all's claims about ASR seems to be that you still don't know what we do there.    

@cleeds 

Amir & Co. believe audio equipment can be evaluated without listening, so sometimes they don't bother to listen at all - blind or otherwise.

Those who clamor most loudly for blind testing rarely undertake such evaluations themselves.

As I just post, there are plenty of blind tests performed at ASR.  And tons and tons of listening tests by me.  I listen to every speaker, every headphone and every headphone amplifier and dac combo.  That is nearly half of what I review.  A count that dwarfs what everyone else does considering that I review about 250 to 300 products a year.

Really, if you are a fan of blind tests, then you should perform them and do so properly.  If you are not, then don't complain about how much we do it.  After all, you are not supposed to care.

Note that I am not demanding anyone go and perform blind tests.  They are time consuming.  But if you are going to make outlandish claims that go against decades of audio science research and engineering, and stomp  your feet that people believe you, then please, do the testing with your ears only.  Do not involve your other senses.  I don't know why this is such a hard concept for some of you to accept.

@amir_asr thank you for the link. Okay one is not a level of frequency and not a confirmation. Has to be repeatable and verifiable. @crymeanaudioriver called the Dutch Boys armatures, this seems quite armature.

One person does not make a quorum. One test is not validating a trend. You did not do the test.  

amir_asr

... I listen to every speaker, every headphone and every headphone amplifier and dac combo. That is nearly half of what I review ...

So you listen to "nearly half" of the components you "review." That’s not very impressive.

... if you are going to make outlandish claims that go against decades of audio science research and engineering, and stomp your feet ...

You seem to be the one who’s foot stomping and (as your bold face suggests) shouting, too.

As for "outlandish claims," it’s hard top your claim that you’ve "reviewed" components that you haven’t ever heard. You may be the only "reviewer" with that level of confidence. Truly odd.

@fleschler 

@Amir Keep your fans on ASR. 

I have no control of anyone.  But if you want us to stay away from this site, then don't create a thread with such falsehoods and continue to repeat the same.  For heaven's sake, you confused me with another youtuber in your introductory post which repeats one every page of this thread.  And continue to misrepresent what we do.

Can't hear differences in cables that measure the same=something is very wrong here. 

Another falsehood.  Cables all measure differently.  I don't know about you but I don't stick a cable in my ear and listen to it.  I listen to the final output of my system.  There, measurements show the same results with different cables.  And it is this testing which I perform which damns them and other tweaks.  That the final sound waves coming out of an amp, DAC, etc. does NOT change when you apply these tweaks.

If you or someone else still finds that the they sound different, it is time to only use your ears.  Don't involve any other senses.  And repeat the test to make sure you can consistently tell these tweaks apart.  

One doesn't have to believe any online person; however, based on the enormous "subjectivist" opinions, people actually do hear differences, right or wrong, personal preferences included, regardless of measurements and decide to what sounds best to them, in a system, in a room.  Nothing you say will change that. 

That's wishful thinking on your part.  Facts are that ASR reach is twice of this site despite being much younger site.  That is not because I am more handsome than  you although I like to think I am!!! 😁 It is because people see the value we provide in cutting through personal opinion spread by company marketing and random joe online.  

I listen in the dark with all lights off when I want to immerse myself in the music, just like a movie theater.  I don't see differences in sound, I hear them.  

Not really.  You are sitting there worrying if this or that other useless tweak like a cable makes a sonic difference.  I and others on ASR do not.  We have confidence in engineering and solid research and testing of what matters in audio, and what doesn't.  Our enjoyment of music then is far greater because we are freed from the wild west of "everything matters" that you live in.

Dismissive of older equipment not measuring as well as cheap current equipment is as wrong as dismissing older recordings because they used older technology with distortions and reduced resolution, etc. of tape versus superior hi-res digital. 

What are you talking about?  I test plenty of "older" technology.  

You think you are superior to nearly everyone who posts on Audiogon because your opinion is based on measurements (except when you don't find a measurable difference ala cables, tweaks, etc).  The overwhelming majority do not think so.  I am just one music lover with recording/performing experience and a high end audio reproduction system.      

Oh, you are now representing this entire forum?  Really?  The segment of audiophile market you represent is tiny.  The desktop audio market dwarfs the rest of the industry like nobody's business.  You think there are millions of people who go and buy expensive audio cables???

As to your final statement, no, you are obsessed with things in audio that don't make an audible difference.  You spend energy seeking variations there instead of enjoying  your system.  

@amir_asr , +1, I think you have been more respectful in your latest posts on this site.

I completely agree with you and appreciate that you added the emphasis:

That aside, I am NOT a scientist.

Feel free to post in some other threads here as many of us can’t even start these threads on ASR, they might be considered controversial or even heresy.

I have no control of anyone.

You control everyone at ASR and we have a consensus of members here who feel you heavily censor your site and bannish contributors on a whim.

You are obsessed with stuff that makes no audible difference.

So are you, you measure and comment on stuff that makes no audible difference all the time.

@cleeds

As for "outlandish claims," it’s hard top your claim that you’ve "reviewed" components that you haven’t ever heard. You may be the only "reviewer" with that level of confidence. Truly odd.

Amir will measure the actual signal output when testing things like cables, and show that they either do not change the signal at all, or that the changes are so extraordinarily low they are well in to the realm we need instruments like a Precision Analyzer to detect them. Remember: instruments are typically devised to extend our limited senses, to reliably detect what we can not detect using our senses alone.

And yet audiophiles will still claim to hear sonic differences with such products, and simply dismiss this evidence. This is akin to dismissing the scientific evidence for the limits of human hearing and claiming "I don’t care what the objective data say, I can hear up to 30 kHz!" While also refusing to take an audiogram with a professional audiologist because "I reject the use of such blind tests to tell me the limits of my hearing."

And you want to talk about "outlandish claims?"

In many such cases...even though Amir actually will include listening tests...he doesn’t need to. Based on plenty of research in to the limits of human hearing, distortion levels, masking thresholds etc, he can say "This is inaudible" from the measurements. To say it IS audible is going against most of the science and engineering theory. So Amir is not making the outlandish or extraordinary claim, the audiophile who claims to hear differences is doing so. So IF you are going to reject the measurements and the inferences from those measurements, these audiophiles could step up and show, in tests where they aren’t peeking at the gear, they can actually reliably identify the differences they claim.

Unsurprisingly, this virtually never happens. But...they’ll just continue to throw darts at Amir and ASR, and continue to brag about having "more capable hearing" than poor Amir and the ASR crowd, without ever putting this to a test.

How convenient.

Those who clamor most loudly for blind testing rarely undertake such evaluations themselves.

@cleeds , I have seen 3 people most loudly clamor for blind testing in this discussion. Amir, who stated he has done many many blind tests. Prof, who has done at least several blind tests, which he described in this thread, and me, who has also done blind tests, which I noted.  How do you think I reached this spot?  I used to believe I could hear differences until it was shown to me I could not. I also did medical research and lived blind testing.

Thomas Edison did blind testing using whole audiences in the 1900’s to show off his new phonograph.  He had an opera singer and his phonograph behind a thin curtain on stage.  The audience was asked to guess if the singing was live or a recording.  The audience couldn’t tell the difference. 
‘Now, do you really believe his 1908 phonograph was as realistic sounding as a live human voice?  It did have something like 110 dB dynamic range. 
Test conditions make all the difference in blind testing.  

@kota1

You control everyone at ASR and we have a consensus of members here who feel you heavily censor your site and bannish contributors on a whim.

ASR has possibly hundreds of active members and dozens of threads, without checking. Such power of control that you attribute to Amir! The bloke must never sleep.

There are here a few disgruntled folk here who feel entitled to have their say on a forum about which they are shocked, horrified and mortified.

I’d bannish myself on a whim were I to insist on pushing the general guidelines to such a point that I became the antipathy of what the spirit of the forum is intended to convey.

I made an innocent mistake on an unrelated forum a while ago and was banished for making light of a certain person in high office. I was unaware of certain strict protocols.

I own my mistake, however innocent it may have been.

As to the Benchmark L4 versus the CJ pre-amp, one of the comments was The LA4 certainly isn't "lacking" anything. It's just transparent. But I could say it does 'lack' certain things in the sense of comparison to the CJ tube preamp.  Based on my experience with both brands, I found CJ to have a house sound, not one that I or my friends prefer.  The Benchmark, on the other hand, stock is clean and clear but lacking in areas the CJ excels-body and warmth of expression.  

There is a cure for the Benchmark.  Replace the computer quality, cheap-ass regulators, power and filter caps, maybe parts of the audio board, the Op-amp, etc. with audio (much more costly) parts.  Benchmark products are only okay stock but can be great when modified.  Their HDR-1 DAC is a great example of a terrific design with mediocre execution.  It is a giant killer when modified.  

You can't help yourself to defend yourself on an open forum, unlike your own ASR. 

Why wouldn't Amir defend himself?

If someone was defaming you on another public forum, wouldn't you think it reasonable to defend yourself?  It makes sense to defend yourself directly to the criticisms in said forum, given people are mostly disinclined to come over to ASR to take a nuanced view of what goes on there.  As we've repeatedly seen the forum strawmanned in this thread.

Do you expect to just be able to publicly  criticize anyone you want, and with any level of misrepresentation, without any consequence or pushback?

@fleschler , would it be safe to say you are not an electrical designer or electrical engineer? If so, under what authority do you make the following comment?

There is a cure for the Benchmark.  Replace the computer quality, cheap-ass regulators, power and filter caps, maybe parts of the audio board, the Op-amp, etc. with audio (much more costly) parts.  Benchmark products are only okay stock but can be great when modified.  Their HDR-1 DAC is a great example of a terrific design with mediocre execution.  It is a giant killer when modified.  

Maybe I'm doing a good thing getting you to waste your time here (as well as @crymeanaudioriver who has more time to excoriate us here as well as on ASR).  

Again with with lies @fleschler . I have now written, I think 3 times, that I do not participate on ASR, and have not in some time. If you have to resort to lying to make your point, I can only assume your position is weak, and you have internal conflict. Please see my Jung quote much earlier.

Is this becoming a right to free speech discussion?

There is a right to remain silent when confronted with an angry crowd, as some may know goes back a couple millennium.

@crymeanaudioriver 

If you have to resort to lying to make your point, I can only assume your position is weak

"Appear weak when you are strong, and strong when you are weak" - Sun Tzu, the Art of War.

For the crowd here that  are blind testing advocates get over it:

@fleschler , would it be safe to say you are not an electrical designer or electrical engineer? If so, under what authority do you make the following comment?

Yeesh.  I hadn't seen that post about the Benchmark.  That's particularly telling....

It must feel nice to feel better informed than truly technically proficient designers, because you have Golden Ears.  Someday I'll reach that super power....

@kota1

For the crowd here that are blind testing advocates get over it:

Did you actually read that article? It doesn’t make the point you seem to think it makes.

It re-enforces the liabilities of sighted listening. In other words, even taking the article's argument as given: sighted listening has ALL the liabilities cited in the article PLUS the addition of sighted bias.

My personal preference is getting the opinions of people who bought gear, used gear, and then took a moment to post a review of the gear. I combine both professional reviews as well as user reviews. The cable cranks will blow a gasket about a cable that costs a premium even though they don’t own it, never tried it and never will. Yet you go on Amazon and see a hundred of five star reviews from people who loved it and are now fans of the brand. I’ll take a hundred 5 star reviews with positive feedback over one cable crank any day. Plus if I don’t like it. refund that bad boy. Whether you find value at $10 a foot or $100 a foot as long as it is worth it to you, NP. 

Quick show of hands, everyone here who conducted their last blind listening test with a PANEL of TRAINED listeners, say aye. Congratulations if your every audio purchase gets judged by your panel. See the link I posted above:

"If you want to take the variable of the listener out of the equation, then your listener panel needs training."

When I buy new gear I get a 30-60 day return period. I drop it in the rack, let it break in (I know, the horror) for a few weeks and then take it out. If I don’t miss it, refund. If I can’t live without it "grinrictus" sets in and I buy it.

@kota1 ,

Did you even read the article you just linked on blind testing? You post reinforces what Amir has said on these pages. Trained listeners are superior to every day audiophiles in detecting anomalies. That you for reinforcing that message.  The article provided no evidence to refute blind testing.

@crymeanaudioriver

It’s amazing isn’t it?

By the very logic of the article kota1 wants us to accept, he’s not a trained listener.

It would be at least as damning of his own method.

PLUS he has added sighted bias in to the mix when he "breaks in" and listens to products.

(This happens all the time when audiophiles attempt to pick apart the usefulness of blind testing.  They suddenly get all picky about variables in an attempt to dismiss the tests, conveniently forgetting that the variables would apply to their own method of testing gear!)

Plus if I don’t like it. refund that bad boy.

That policy must be particular to America. In my country, there are conditions on the reasons you may wish for and be provided a refund - not liking it is not one of them.

Such a liberal policy in America has unintended consequences on a number of levels.  But that is what is known as micro-economics and out of bounds on this esteemed forum.

@prof

Why wouldn't Amir defend himself?

If someone was defaming you on another public forum, wouldn't you think it reasonable to defend yourself?  It makes sense to defend yourself directly to the criticisms in said forum ...

The problem with ASR as of late is that such opportunity is not always given. A loud member, or a moderator, or a whole pack of them, publicly accuse an "inconvenient" member of disingenuity, of lack of knowledge, or of low intelligence, and then said member is immediately banned, with no way to defend himself there.

Do you expect to just be able to publicly  criticize anyone you want, and with any level of misrepresentation, without any consequence or pushback?

As far as I can tell, this is absolutely the expectation of certain ASR moderators, and also of certain ASR members whose opinions are considered as the only valid ones by some moderators. That's the crux of the issue the OP raised.

As a medical analogy, this resembles an autoimmune disease. A healthy, balanced moderation serves a function similar to the one served by immune system. A hyperactive censorship, on the other hand, may be gradually killing ASR organism. 

noske

Plus if I don’t like it. refund that bad boy.

That policy must be particular to America. In my country, there are conditions on the reasons you may wish for and be provided a refund - not liking it is not one of them.

That’s interesting. "Satisfaction guaranteed or your money back" is a common guarantee in the US, although often with a time limit.

The topic about whether or not a vendor should offer refunds is up to them. They have to weigh if the additional sales they make because of the policy outweighs the number of refunds they get. It is a strategy known as "risk reversal". The vendor takes the risk instead of the customer. Let them run their company as they see fit and the customer shops as they see fit. See:

https://ezinearticles.com/?Risk-Reversal---A-Strategy-I-Learnt-From-Jay-Abraham&id=1009021

@fair +1 

A loud member, or a moderator, or a whole pack of them, publicly accuse an "inconvenient" member of disingenuity, of lack of knowledge, or of low intelligence, and then said member is immediately banned, with no way to defend himself there.

@fair

The problem with ASR as of late is that such opportunity is not always given. A loud member, or a moderator, or a whole pack of them, publicly accuse an "inconvenient" member of disingenuity, of lack of knowledge, or of low intelligence, and then said member is immediately banned, with no way to defend himself there.

I don’t have the lengthy experience posting on ASR that you have, but I’ve certainly seen new members of subjectivist bent banned very quickly. And to put things in perspective, I’ve seen that many more times than I’ve seen an actual single- or double-blind test performed for or by ASR.

@amir_asr

There are very frequent blind tests posted by members. Here is a very recent one:

Amir uses a different dictionary from others "frequent" here means "every now and then".

That aside, I am NOT a scientist. Never claimed to be. Never want to be. I follow science just like your doctor does.

Perhaps the site would be better named Audio Medicine Review? 😊

@laoman

Here is the thing. A while ago I was looking around for a new DAC. I listened to about 8 and in no case was the cost of the DAC mentioned. I just sat in a chair and listened to the same music. One that I disregarded very quickly was the Topping.

While ASR-ers don’t actually perform DBT or similar nearly as often as they demand it of others, it is useful to consider what it is and why we should do them sometimes. If we take the DAC example, imagine if I were to do comparative listening test for DACs including a Topping? Now I’ve read ASR for a good while and watched enough of the videos so when I think of Topping I think of Amir. So how can I listen to a Topping knowingly and enjoy it? To give it a fair audition, I’ll need to disguise the brand.

My My …

“ But if you want us to stay away from this site, then don’t create a thread with such falsehoods and continue to repeat the same.“

Only a totally  self absorbed Narcissist would be so stupidly vain enough to admit to his and his familiars raison dêtre in posting on this forum.

@tsushima1 +1

@prof, I ceased to be amazed, many hundreds of posts ago.

@tsushima1

Only a totally  self absorbed Narcissist would be so stupidly vain enough to admit to his and his familiars raison dêtre in posting on this forum.

Indeed, sublime post and basic insight in one.

Said narcissism used to take my breath away, but I ceased to be amazed hundreds of posts ago.

@noske ,  "Fanaticism is always a sign of repressed doubt. ~Carl Jung"

@cleeds

That’s interesting. "Satisfaction guaranteed or your money back" is a common guarantee in the US, although often with a time limit.

Not really interesting. Consumer law guarantees certain rights to both vendor and buyer which cannot be denied for a year, maybe two.

The statement you quoted is a business decision by the vendor - time limit 30 days or so? Who pays for all the fuses that are returned within that time period, because they didn’t like them?

Either you make a product that people wanna keep for the duration of their expected lives, or refund them if it is faulty.  Not simply because ya tried and didn't like - that is such an expensive business decision whereby the consumer takes on the risks of other consumers before him who were didn't like the fuse.

noske

Consumer law guarantees certain rights to both vendor and buyer which cannot be denied for a year, maybe two.

Not in the US. It's perfectly legal to offer a "cash and carry, as-is, no refund" policy.

My My …

“ But if you want us to stay away from this site, then don’t create a thread with such falsehoods and continue to repeat the same.“

Only a totally  self absorbed Narcissist would be so stupidly vain enough to admit to his and his familiars raison dêtre in posting on this forum.

What warped sense of morality do you have where someone defending themselves from falsehoods and lies is the bad guy?

The statement you quoted is a business decision by the vendor - time limit 30 days or so? Who pays for all the fuses that are returned within that time period, because they didn’t like them?

With a margin of likely well over 1000%, I don't think some returns or even a lot of returns is an issue. Even 50% would be totally acceptable.

Either you make a product that people wanna keep for the duration of their expected lives, or refund them if it is faulty.  Not simply because ya tried and didn't like - that is such an expensive business decision whereby the consumer takes on the risks of other consumers before him who were didn't like the fuse.

Amazon's return rate I think is close to 20%.

 It's perfectly legal to offer a "cash and carry, as-is, no refund" policy.

Now we are off topic but with the grace of the mods - that policy is common for used goods, of course. 

I speak only of new items.  That narrows it down. If I sold you a new IPhone with the no refund policy I'd be breaking the law of my country,   Which is not America.

While ASR-ers don’t actually perform DBT or similar nearly as often as they demand it of others

If a DBT is done, you have to count each test, no matter the number of participants, as a single data point. Because the test results may be biased. So, the number of tests required to be statistically significant goes well past the majority of tests ever reported. Make the sample of tests large enough to be statistically significant and you are at the man on the street level of participants, not trained listeners. And that is how you get Bose level of audio quality. 

@amir_asr makes a lot of assumptions. Assuming what ones ideas on audio are. Amir I for one will never point a browser at your site nor click on a link you post.

You can assume I am not interested in your findings on audio equipment I have zero interest in. You are correct you are not a scientist nor an engineer. Nor are you a professional reviewer, you are average joe with some software and that is it.

No need to respond as someone here said you monkeys will respond I could care even less about there opinions. 
 

Blind testing at Harmon/Infinity (PS- Don’t try this at home kids!!)