Based on your "I hear it when its done by the CD player but not when its done by the preamp," I believe your observations may be purely psychological manifestations. Either that or the phase invert button on your preamp is nonfunctional. Phase invert or phase reversal should both do the same thing, and doing it twice is like not doing it at all.
So, it shouldn't matter one iota whether polarity is inverted by the recording engineer that did your CD, your DAC, your amp, your preamp, or by reversing the speaker wires on your speakers.
Either you have established absolute polarity or you are reversed. Either the black wire connects to the red terminal and the red wire connects to the black terminal, or vice-versa. Someone correct me if I'm wrong, but there is no halfway ground here...
So, it shouldn't matter one iota whether polarity is inverted by the recording engineer that did your CD, your DAC, your amp, your preamp, or by reversing the speaker wires on your speakers.
Either you have established absolute polarity or you are reversed. Either the black wire connects to the red terminal and the red wire connects to the black terminal, or vice-versa. Someone correct me if I'm wrong, but there is no halfway ground here...