The musicians may not know what it sounds like from various seats in the concert hall. Nor what it sounds like in venues they have not performed in. Nor concert musicians what it sounds like in a recording studio and vice versa. Nor musicians recorded in Abbey Road studios what it sounds like elsewhere.
But anyone who cares, has normal hearing capabilities, and listens knows what it sounds like at that time in that location with whatever the source may be.
Is that good enough to assume we do know what music sounds like? If we don’t, so what? I don’t know what it sounds like on Mars. Won’t let that bother me too much. Nor if someone else claiming golden ears tells me I don’t know what music sounds like.