Crossing Left and Right Speakers

There sound should be no different other than reversing the channels of the recorded output left to right and vice versa. 

Unless-

1) Something was inadvertently wired out of phase. 

2) The room acoustics affect the sound of the left and right speakers differently- e.g. reflections, absorbtions, reinforcement of different frequencies, back of room differences- e.g. one side open, one side closed off etc. 

Test this by reversing the output interconnects of the source channels.  If the sound is the same as reversing preamp outputs then it is because of room acoustics. 

@jimmy2615 

I checked and the records I heard the issues on, only one was Binaural.

I'm wondering if you are listening to familiar music. Could it be that you are not hearing what you expect?

does the reversed image sound the same if the amps are non-reversed and you turn your head away from the speakers? 

It's hard to believe you get anything other than a mirror image.  However, I do see the point about familiar music and the room not being symmetrical.  I can imagine that your ear is trained for a specific image and it would seem like a different room if the channels were reversed.