Wow, how frequently words fail us. Most test records use the word "phase" to deal only with whether both speakers share the same positive and negative connections throughout all that goes before them. Usually if you experience the signature disconcerting location-less sound, one speaker's connections are wrong, the red connection on the speaker wire is connected to the black binding post.
Polarity involves whether what you get in the drivers moving outward when the microphone at the recording retracted as the sound impacted on it. This would work only were recording engineers meticulous in wiring all microphones the same. Since few are concerned with this, what you hear is unlikely to have similar polarities across all microphones. All you can do is to use your polarity button the way it sounds best on each record.
This is further complicated by how your manufacturer accomplishes polarity change. Most do so by adding another stage of amplification to invert the signal. This usually sounds worse than the preamp without this additional stage.
