Measuring DC offset.

That DMM is showing a couple of different things.   First, with an open input you are probably picking up some 60 Hz noise.   Try shorting the input and see if the fluctuation changes and DC offset settles down to either a positive or negative number.   Capacitive coupled outputs have a bleed resistor to ground to shunt the initial charge current to ground.   Depending on the cap, it can take quite some time for the leakage to drop to zero.   Any polymer cap, that is polyester, polypropylene, etc., will have zero leakage but will have a charging current that should dissipate in a short time.  Electrolytics charging current and leakage current is much the same since their leakage is also the dielectric stabilizing under a charge.   Consequently, they can have leakage current that can last for many hours.  I have measured leakage in good quality electrolytics that took over 24 hours to subside.   The cheap ones can take several days.   Do understand you can also have an electrolytic coupling at the input as well as the output, which will get amplified and appear at the output.  In general though, the numbers you are reading are nothing to worry about.

The numbers sound fine.

Unless it's a tube amp. That would be a weird one.

I like the DC Offset to be under 10mV but depending on the specs that might be nominal.