Do you all understand that the metadata and music data are not in the same places in the file or is that ... not relevant to this discussion?
The metadata, if read at all, is not interwoven with the music stream. It's a completely separate part of the file, and from a performance issue, adding/removing files from one folder to another in a hard drive is no different than adding or removing another software program or game to your hard drive. If these make a difference to your sound quality there's a very big error happening somewhere.