Should a reference speaker be neutral, or just great sounding?

My thoughts on what constitutes a reference are similar to those from @kr4  above.   It is something that is well characterized and whose attributes are not in dispute.  Thinking in simple terms, it is like a graduated ruler, against which one can measure the length of multiple objects with confidence.  Use of a ruler is not context dependent in a practical sense.  I can use it at my desk, in my living room, in my listening room and it works the same.  I can use it in Tennessee and travel to Leipzig and it still works the same.   

I think of "reference" audio equipment and speakers in particular primarily within the context of use by a reviewer.   I don't think of any of my pieces as being references even though when I audition new gear I use my existing equipment as a reference of sorts.  I think more in terms of "destination" pieces.  Those pieces that are so good I will probably never replace them.   I regard my Atma-Sphere M-60 amps are destination pieces.  Possibly my Coincident CSL line stage.  I would be hard pressed to significantly improve on the M-60/CSL pairing.  Future changes to my system will be to source or speakers, or perhaps to cabling or isolation devices. 

Speaker performance is so tied to how well suited it is for the room in which it is used and also to being fed by a well matched amplifier that I just don't know how one really could have a reference speaker in the absolute sense.   A reference should not be context dependent.   What good would a ruler be if it were made of a material that had a high coefficient of thermal expansion?    One that would only give reliable readings in a room where the temperature was rigorously controlled at 20C?  That context dependence is the nature of speakers.  How they perform is so context dependent I just don't think it is possible for them to act as true references. 

When a speaker manufacturer refers to their product as a "reference", I would expect it to mean that all the boxes are checked; the specs are great, the parts and materials are very high grade and the SQ at a level that would satisfy most critical listeners. Neutral in at least the sense that no tonal variation or other negative attribute stands out. Unfortunately, the accompanying price tags often make all this irrelevant to me as I can’t/won’t pay the price of admission.
For the rest of us unwashed masses, we have lived with speakers long enough to become acutely aware of whatever shortcomings we judge them to have and seek to address those issues when it comes time to replace them.
So, if you select your next speakers with your ears and not based specifications, reviews or online advise, your reference is probably whatever speakers you have now.

“I guess I always thought of "reference" as being similar to laboratory grade, like a precision scale”

When we can measure what makes a speaker a good listen, it will be.

I prefer speakers with a pronounced estuary accent.

DeKay

A reference is anything you refer to. For words we can refer to a dictionary. You could look it up and see, that is literally the meaning of reference. Nothing about neutrality, objectivity, or anything like that. Your reference could be a Picasso painting of a woman with three boobs and a few other odd body parts. Your reference could be last years iPhone. 

A reference is nothing more than a standard. There's nothing objective about it, not at all. A meter is a reference, defined about as objectively and with as much precision as anything probably ever could be. Yet the meter itself is totally arbitrary. We just decided for convention to make it what it is. 

Same with your speaker reference. Totally arbitrary. Make it what you will.