Every one is correct. There will be no harm done. But you will, of course, lose some of the lowest bass since that comes from the port. How much loss depends on how much you stuff. Experiment; try using various amounts of say Dacron or plastic foam, etc.
If you stuff it so there is no perceptible bass from the port you will also improve the Q of the bass; what bass you have will be tighter.
An old, fine example of a stuffed port is the Dynaco A25 loudspeaker, the best selling speaker of all time. It used a stuffed port and while the bass was not subterranean, the quality was excellent. SEAS of Norway built it for Dynaco and have designed a modern kit version, the A26, which Madisound sells as a kit.