Audio Science Review = "The better the measurement, the better the sound" philosophy

@axo1989 

But when it comes to amplifiers, ASR rarely listens, with the general justification that magnitude of differences are too small to differentiate (with some exceptions). As you can see in this thread all listening (that isn’t done by ASR with partial protocols) is routinely dismissed as sighted bias, expectation bias, focus bias etc (you too can play bias Whac-A-Mole). 

The pushback like that is made when the claimed sighted tests go counter to solid body of engineering and research.  Say a power cable improves the sound because you swapped one for the other and proof of being right is "I have been an audiophile for 30 years" and you rightly get strong pushback.  Don't offer such as proof point and you are generally fine.  And even if folks object, you should be cool because you weren't going to prove anything.

How else do you want us to behave?  I once asked my doctor if he could help me with research into weaknesses of blind testing.  He just about threw me out of his office!  He said, "Amir, the foundation of what I do is based on blind testing; I can't participate in any attempt to cast doubt on it."  It wasn't my intent to cast doubt but I fully understood his position and continue to see him.

Before funding ASR, I was the co-founder of Whatsbestforum (WBF).  We thought by allowing both camps to state their position, life would be good.  Well, it turned out to be anything but.  The conflict eventually crept between me and my partner and I sold out my shares and got out.  I decided then to go the "pure" route and start ASR.  The name clearly states that we are committed to teachings of audio research and engineering for decades.  We don't pretend to be smart enough to invent our own rules of universe for audio and champion that to everybody else with vengeance as folks are doing here.

Net, net the response you mentioned is what you should expect if you come and make outlandish claims. You have seen me respond similarly here.  As I said, it is a jazz club and you shouldn't expect folks to take kindly to you demanding that you play country music.

You want to come and challenge our position? Do so with solid research and science driven listening tests.  Be ready to defend yourself and not cry victim with "oh they ask for controlled test and tell me about bias."  Of course we do.

Indeed, many people who have a short life in ASR mistakenly assume they are stating something new to us that we are just going to roll over and accept.  Member @kota1 for example shows up and says every cable needs to be broken in for 100 hours or the test is invalid.  We have heard these claims a million times.  Don't be the million and one member who thinks you should just throw that at us and we go, "oh, I didn't know that; thank you for that information!" 

Read the forum a bit and get educated on what and who we are.  Then participate if you need to.  You are welcome to challenge us on every topic.  Members do that to me all the time.  But be ready for heaven's sake with some back up worth more than a fortune cookie paper!

@axo1989 

One the one hand, Amir does ignore Toole’s advice that speakers should be listened to comparatively for evaluation to be meaningful. Even something as simple as setting up a curtain and turntable and enlisting helpers from his cohort of followers is dismissed.

This was a planned activity from day one that I started to test speakers.  Indeed, i have held on to a mountain of speakers for this very purpose.  But you may have heard of a thing called the pandemic.  Our local audiophile group where I was hoping to conduct such tests stopped meeting (and went virtual) so the project is on hold.  Meanwhile, one member did post such a comparison: 

And a much more sophisticated one using a turntable was created as well:

I highly encourage such efforts.  I provided speakers for the second phase above to the organizer and happy to do so for anyone who likes to conduct them.

Such testing is extremely time consuming.  But good news is that anyone can do it.  You don't need my experience or instrumentation.  So no one should be waiting on me for it.  My time is best spent providing objective data such as measurements.

To position this as me against Dr. Toole's teachings is very much out of line.  Nothing remotely is true in that regard. I simply don't have the resources or time to do this kind of testing on every speaker that lands here.  

My listening tests in reviews is provided on "as is" basis. I do them because if I didn't, I would get more complaints.  "Oh, he doesn't listen."  I have tried to make more sense out of them by developing the EQ technique.  The outcome there has been quite positive with many trying my EQ profiles and liking them over stock performance.  If folks want to ignore them -- and many do -- it is no skin of my nose.  I perform them because I am curious myself how the measurements translate into sound and a form of listening training.

@amir_asr  "I am not in audio science.  I am posting here.  Ask a pedantic question, you receive an answer in kind.  :)"

You have been very generous with your time, I'm sorry I am testing your patience, but I've am seriously curious how you inject price or value judgements into your reviews.

@tonywinga 

Some of the greatest sounding musical instruments, in fact most all musical instruments  and even concert halls were designed and built without computers and electronic analysis equipment.  They were designed and built by artisans with skilled hands and ears.  It's all about what we hear.  When someone tries telling me I am not hearing what I think I am hearing, well that goes over with me about like putting a tax on a child's piggy bank.

Once more:  listening tests are the gold standard in audio research.  No one is telling you to substitute measurements for it.  

What we say is that don't go believing marketing claims that have no verification with controlled testing, or make sense at engineering level.  We prove the latter with measurements.  Company claims the power conditioner lowers your audio system noise?  Well, we measure that.  If the result is that noise has not changed one bit, then you know the claim was wrong. 

Why is this so odd for the few of you to accept?  You say your local water is making you sick?  Folks come out and measure to see what is in it.  If it is pure and clean, then that is very important information. 

Importantly, don't confuse creation of art with replay of it.  Our business is the latter. The two are completely different universes.  Audio equipment should NOT be in the business of creating or modifying art.  If it is, then it is not high fidelity.  And will impart the same signature on every music you play -- something I dislike dearly.

As to what you think you are hearing, that is NOT in doubt.  What is in doubt is what you say it means when you did not block all other senses than your ear.

My listening tests in reviews is provided on "as is" basis. I do them because if I didn't, I would get more complaints.  "Oh, he doesn't listen."  I have tried to make more sense out of them by developing the EQ technique.  The outcome there has been quite positive with many trying my EQ profiles and liking them over stock performance.  If folks want to ignore them -- and many do -- it is no skin of my nose.  I perform them because I am curious myself how the measurements translate into sound and a form of listening training.

That all makes sense to me.  Given the range of audiophile viewpoints you can't please everyone.  If you don't listen, you'll get complaints about that.  If you do listen, you'll get complaints that you aren't using a rigorous enough protocol.  If you use a rigorous blind protocol, you'll get pushback from the anti-blind-test faction who think blind tests obscure results and you should have listened "in relaxed sighted conditions, like a normal audiophile."

Personally, I think that yeah, blind listening to speakers would hue most consistently to the ASR remit.  But your compromise of  "here are my impressions take them or leave them" seems a reasonable compromise.