In my view, it is one more thing to obsess over. I think it's futile to sit there and attempt to identify the best polarity setting for every single track you will ever play.
I have noticed differences in polarity - only by accident. Once I was listening to my favorite orchestral recording and about 45 minutes through it I was thinking something's not quite right. I chalked it up to a bad listening day. But when I powered down, I noticed that the polarity switch on the DAC was at 180 instead of the zero I keep it. Son of a bitch, I thought, I would have not heard the difference if I actively A/B'd it - I know, I've tried when I first got the unit. To me, it just can't be done in short bursts. I think this is a fundamental given of trying out new equipment - it has to grow on you before you notice anything different. I've had my Marantz SA11 for six months now and I can't settle just yet as to which of the six possible filter settings on CD or of the three for SACD work best. Too many recordings to go through.
This is the same way I noticed other things off setting - whether it was a bad tube, a damaged tweeter, different switch setting on the power conditioner or a change in the DSP setting on a Digital Lens. It took a good length of time when I was not expecting it.