Wouldn't it be better at the speaker end since that would minimize the amount of inverted signal? :)
In my integrated amp manual it says that absolute phase is inverted and that you can correct by inverting positive for negative at the amp. It goes on to say that the phase of recordings is indeterminate so no matter which way you choose, you will be wrong about 1/2 the time anyway. So does it make sense that as long as you have each side set up consistently then it does not make a difference?