To set up your speakers properly you need a sources with which you are totally familar. When your system reproduces these recordings properly you're done (although you can always revisit set up for fine tuning).
Recordings which don't sound right after this set up are just giving you what the engineer did with the recording. One of the benefits (and sometimes downside) of a highly revealing, well set up, system is you hear all the information upstream, which can include some warts. Panels can be difficult to set up properly but the things you are noticing aren't consistent with set up. They sound like recording artifacts to me.
Recordings which don't sound right after this set up are just giving you what the engineer did with the recording. One of the benefits (and sometimes downside) of a highly revealing, well set up, system is you hear all the information upstream, which can include some warts. Panels can be difficult to set up properly but the things you are noticing aren't consistent with set up. They sound like recording artifacts to me.