Ground on a scope is normally chassis ground (on all except fully isolated). Careful when you connect ground both for potential for a short and noise you didn't expect.
Even when scope is fully isolated there will be always some very small capacitance between BNC connector ground and the earth ground. Because of that, probing any circuit that is not battery powered can deliver nonexistent noise, even if probe is shorted on the probed circuit ground. It happens because there will be small current between probed circuit ground and scope's ground causing voltage drop on the probe's return. Scope will "see" this voltage drop as a signal (since input is related to BNC connector GND and not the end of the probe GND). It is just in case you wonder why scope shows anything (very small HF noise) when you touch circuit with shorted probe.