I wouldn’t worry about it but I would be curious. Even if not so loud, the hiss should not be there. I am allergic to anything that gives more digital glare, and the hiss may be a suspect, even if not much notable by itself when playing music in the listening position. So I would wonder, why does this happen. Does the DAC behave that way with other amps also? Are you sure other sources into the amp don’t give the hiss? You’ve said its the same from xlr and single ended inputs. If you have switchable inputs between sources on your amp - the same across these, also? The fact that the hiss does not vary with the volume level makes me pause. Something wrong in the DAC? Or mismatch, amp / DAC?
I just tested in my secondary system, with a Ming Da integrated amp, with a few small DACs including the Fiio x3 and Huawei p30. Aurum Cantus speakers, with top notch ribbon tweeters well able to tell me what goes on. No hiss, but some hum. Varying with volume. Below the "worry" border, though - only heard clearly at max volume.