If you have to move them then you have to move them.
The biggest consideration for these setups is to make sure that there is no rocking of the speaker at all. You want to be able to press on the top front to back and make sure there is zero tilting going on. Also, you do want to use locking casters.