I like to leave my grills off, I find they use too much gas when kept on. It only takes a minute or two to warm up, not like the stereo which takes 15 or 30 minutes.
I do leave the grills on the speakers, can't say I notice a difference with or without.
Oh, and we have a big black and white cat