Take a look at Vanatoo or similar maybe even lower cost. Whatever your budget, I would keep it as simple and cost effective as possible for a 4 year old. If he tires of it before long then at least you have a nice sounding and flexible second system to play with moving forward.
As a point of reference, FWIW, when I was ~ that age, maybe a year or two older, in the mid 1960’s, I started to become interested in music and hifi and had my sisters portable Magnavox record player and records to listen to. SHe was in high school already at that time. I was as much interested in the record player as the music and I received a few electric shocks along the way as my curiosity went unchecked.
I often wonder if those electric shocks I received doing things with electric gadgets I probably should not have been doing at that age made me what I am today....... a big time hifi and music affectionado. YOu never know.... :)
A little kids keyboard is an interesting proposal. You never know what direction a kid at that age might go. So if it were me I would address his interests cost effectively but not go overboard at that age. and see what happens.
If he starts demanding high end hifi fuses before full adult hood then you know you might have a problem.