What does a 'refined sounding' speaker mean ?

Could mean a lot of things but typically it has to do with timing. A more refined speaker would have natural tone, a little more detail & speed, as well as better attack & decay of individual notes. 

It probably doesn't call attention to itself or any part of the audio spectrum, does everything effortlessly and naturally.

We can guess what it means, or we can turn to the great audio reviewer Lewis Carroll who explained it so clearly in his great audiophile tome Through the Looking Glass:

"When I use a word," Humpty Dumpty said, in rather a scornful tone, "it means just what I choose it to mean—neither more nor less." "The question is," said Alice, "whether you can make words mean so many different things." "The question is," said Humpty Dumpty, "which is to be master—that's all."