Best investment I ever made for my son was a keyboard teaching toy. Infinitely patient, he would play with it for hours a day then set it aside for a few weeks, then pick it up again. Highly recommended.
As for amps & speakers, nothing you list will really be suitable for several years, until he can read and has a phone, so get an Amazon Echo Show 8 and connect to a pair of powered speakers via the aux out port and a Y-cable. Maybe a pair of Audio Engine A2 or A3 or Kantu YU4 or YU6 You should be able to wrap it all up for under $500, and he’ll have quality music, voice control, lyrics on screen, parental controls (more important than you think!), and a bunch more. And you or your wife can shut it off from anywhere with an Echo, or from your phones.