Do I need to turn anything off while I'm doing this?
Yes, definitely, although it might be less critical if a different input were selected while you applied or removed the short. But that would be bad practice anyway.
If a component like that is bad, shouldn't the buzzing be present constantly?
Probably, but not necessarily. The component could be marginal or intermittent.
Regards,
-- Al