Typical bias current for LM4562 is 10nA. Taking into account possible unbalance of 10k we get 0.1mV unbalance.Good, keep going. Calculate the effect of the input offset current based on the higher of the two source impedances, and add that. Then add the input offset voltage. Then multiply by the the sum by the DC gain of the circuit . . . you now have the number you're after. A mV or two of offset gets pretty scratchy-clackety on the source-selector and volume controls of a following preamp (provided they're DC coupled).
Again, it COULD work fine if you change to DC output coupling to accompany your opamp sub (10nA is pretty low for a bipolar opamp) . . . but like any modification, a full analysis of the particular circuit as it pertains to your particular mod is required if you want to be confident of a positive outcome.