Why do I need a switch?

If you have enough ports on your router, you don't need a switch.  Of course, listen and make sure you are enjoying your music.  FWIW, I have a switch between my router and my streaming device, but only because I needed extra ports in the room.

English Electric 8 switch
 

This is a really good switch and I was lucky to have found and bought it for about 600 bucks Plus they give you a decent Quality shorter ethernet cable and the connecting power cord is of very good quality. 
 

It's an elegant looking switch and the blinking lights are very subdued. It's built very well and all the underlying technical rationale is very compelling.

So the TP-Link boxes are installed and everything seems to be working,  but I really don’t hear any difference. I’ll give it a few more days to break in or whatever and if I still can’t hear any difference, back they go.

So you added optical fiber in the Ethernet chain from your router to your music server?  Anything else between the router and your music server?

And you hear no difference?  Well, I’d say that you should have heard a difference immediately if you were going to hear anything.  It’s likely that it will not change over time.

OP,

Thanks for documenting your experimentation.