It is estimated that about 50 percent of recordings are recorded in reverse polarity... so about half of your recordings should sound a little better with the polarity inverted.
Phase is frequency dependent as Kal said, and it is used by many as a synonym for polarity, which is not correct.