I think you’ve already nailed it. It can be virtually impossible to confirm what the actual weak link is until you’ve made (usually) at least Some sort of substitution.
Sometimes even rudimentary substitutions can be rather illuminating on a given problem...and sometimes can only cloud the issue further. The real trick is, I think, as you have done, is to try to identify the exact nature of whatever sound problem you feel you have and then to try to work backward, by way of substitutions, toward uncovering what one component item (or more than one...or even several - yikes) that is responsible. Sometimes I’ve found that continuing to make substitutions (for various other reasons) had "improved" the one particular problem I’d had in the back of my mind without ever actually making it go away...only to finally hit paydirt and find the true cause to be elsewhere. Then again, on a different sort of problem, a similar series of substitutions may act to actually audibly unmask the underlying problem, making it even more irritating. In the long run of having futzed around with all that over the decades, Either of those two scenarios, either the unmasking of or the covering up of a problem, can sometimes be useful in the diagnosis process and anything like that can help reduce it all down to the next likely suspect.
But, yes, without a doubt, it is most often like trying to pull at a sweater, especially when things seem to be, as you say, sounding pretty good and pretty well balaned to start with.
The real question may be how much the original problem is bugging you and exactly how far you might be willing to go to squash it. Not an unrealistic or unworthwhile question to pose in this hobby at all, I’ve found.
Sometimes even rudimentary substitutions can be rather illuminating on a given problem...and sometimes can only cloud the issue further. The real trick is, I think, as you have done, is to try to identify the exact nature of whatever sound problem you feel you have and then to try to work backward, by way of substitutions, toward uncovering what one component item (or more than one...or even several - yikes) that is responsible. Sometimes I’ve found that continuing to make substitutions (for various other reasons) had "improved" the one particular problem I’d had in the back of my mind without ever actually making it go away...only to finally hit paydirt and find the true cause to be elsewhere. Then again, on a different sort of problem, a similar series of substitutions may act to actually audibly unmask the underlying problem, making it even more irritating. In the long run of having futzed around with all that over the decades, Either of those two scenarios, either the unmasking of or the covering up of a problem, can sometimes be useful in the diagnosis process and anything like that can help reduce it all down to the next likely suspect.
But, yes, without a doubt, it is most often like trying to pull at a sweater, especially when things seem to be, as you say, sounding pretty good and pretty well balaned to start with.
The real question may be how much the original problem is bugging you and exactly how far you might be willing to go to squash it. Not an unrealistic or unworthwhile question to pose in this hobby at all, I’ve found.