Should a reference speaker be neutral, or just great sounding?

When a speaker manufacturer refers to their product as a "reference", I would expect it to mean that all the boxes are checked; the specs are great, the parts and materials are very high grade and the SQ at a level that would satisfy most critical listeners. Neutral in at least the sense that no tonal variation or other negative attribute stands out. Unfortunately, the accompanying price tags often make all this irrelevant to me as I can’t/won’t pay the price of admission.
For the rest of us unwashed masses, we have lived with speakers long enough to become acutely aware of whatever shortcomings we judge them to have and seek to address those issues when it comes time to replace them.
So, if you select your next speakers with your ears and not based specifications, reviews or online advise, your reference is probably whatever speakers you have now.

“I guess I always thought of "reference" as being similar to laboratory grade, like a precision scale”

When we can measure what makes a speaker a good listen, it will be.

I prefer speakers with a pronounced estuary accent.

DeKay

A reference is anything you refer to. For words we can refer to a dictionary. You could look it up and see, that is literally the meaning of reference. Nothing about neutrality, objectivity, or anything like that. Your reference could be a Picasso painting of a woman with three boobs and a few other odd body parts. Your reference could be last years iPhone. 

A reference is nothing more than a standard. There's nothing objective about it, not at all. A meter is a reference, defined about as objectively and with as much precision as anything probably ever could be. Yet the meter itself is totally arbitrary. We just decided for convention to make it what it is. 

Same with your speaker reference. Totally arbitrary. Make it what you will.