Balanced will always sound louder, but your system isn’t going to be capable of playing louder. So you have to adjust the volumes to be the same for a true comparison of quality.. Fortunately my Audio Research equipment allows that. So I can flip between them with the volume equalized.
The only direct comparisons I have done was with Transparent cables and Audio Research gear (all balance designs). I was not able the discern a difference.
I think in most cases it is negligible or at most a small difference. If the component design is balanced and you use single ended interconnects… then inside the components are converters to change from balanced to single ended and visa versa. In high end equipment, the designer puts effort and high quality sub components into doing it.
XLR cables cost more… often a lot more. If I was relatively young I would go with balanced… component design is headed that way. But if I was on a budget you can save money on single ended. But as has been pointed out to be sure with your components, you have to try to be sure.