Ok so seems you like the Graham line in which case going to the Phantom is probably the way to go. I'd think that $3.5K on the phono side does not get you to the ARC PH8 or Tom Evans Audio Groove + (to quote two great choices on the 'Gon both in the $4.5K range) that would be where I'd like to be -- the choices at $3.5K seems rather "meh" to me and I suspect you'd feel the upgrade bug soon again (of course that's just my opinion, I use the two above as they're both stages I have heard and like, I have not heard all the rest)
In other words I'd optimize from the front end forward, you have the table, you have a pretty nice cartridge now optimize the arm and keep moving forward from that (I've always been a source first guy). I've always believed in pushing as far as you can afford on any one piece and then working on having the rest of the system catch up. It may be that at this point in time upgrading your phono to $3.5K would yield a better improvement than the same on the arm but it could be a dead end whereas the arm upgrade could carry you forward to $10K+ cartridges.
In other words I'd optimize from the front end forward, you have the table, you have a pretty nice cartridge now optimize the arm and keep moving forward from that (I've always been a source first guy). I've always believed in pushing as far as you can afford on any one piece and then working on having the rest of the system catch up. It may be that at this point in time upgrading your phono to $3.5K would yield a better improvement than the same on the arm but it could be a dead end whereas the arm upgrade could carry you forward to $10K+ cartridges.