Four inches should be plenty of clearance for the ports. Even a bit less is still functionally fine - as the port gets closer to the wall the effective tuning frequency moves lower, which is the direction you'd want it to go.
I assume that the controls for the subs should give you a fair amount of adjustability. If the bass is too boomy and you can't fix it with the controls on the subs, you might try stuffing foam or a rolled-up towel into one or both of the ports.
Duke