Can you build a transparent XLR selector box?

No. Just get a line stage with sufficient inputs.
Every additional switch, jack, wire and cable usually adds extra grunge.
Interesting you mention patch bays. Make like one and plug/unplug

I know that the answer is yes, as in every professional recording studio employs patch bays.

So if every professional recording studio does it, then it must be transparent. Interesting.

So if every professional recording studio does it, then it must be transparent. Interesting.

Tongue in cheek

I know that the answer is yes, as in every professional recording studio employs patch bays.

But have you personally tried one in your hifi signal path and were able to notice any effect on your sound?

Its important to understand that most studio equipment supports the balanced line standard also known as AES48. Most high end audio home equipment does not- although there are a few exceptions which are rare. Because if this you'll find that a patch bay or XLR switchbox may well affect the sound.