We’re thinking about this from a human perspective, but if we reach ASI we will be ants in comparison.
My point is precisely the opposite of the one you suggest.
You suggest AGI will be superior to us and will decide. You suggest that we must wait.
My point is precisely that the risk of self erasure of man is almost certain not because AGI will be superior to us but because we will idoloze it as God.
( apart the other serious problem of alignment between a.I. and us
and the more serious distinct problem of alignment between us and the machine
and the totalitarian control over us by corporations using it in a completely digitalized world with no money and social credit as China . You know it is Bill Gates that helped chinese to implement surveillance with social credit digitalization 15 years ago in China, do you ? )
Our social fabric is absolutely immature and ill ...This is an era of incredible challenge.. The only person who observed and described this on a spiritual level was Rudolf Steiner whom i studied extensively 50 years ago reading 200 books...
He described in 1920 all there is about A.I. and even located it in silicon valley which did not exist at the times...
He created organic agriculture and new approach in natural medecine and a new pedagogy and even a new understanding of the threefold social order only to counterbalance the deshumanization of man by techno cultism, as we see it everywhere now..
Waiting for AGI to solve our problem is waiting to die...It is techno cultist attitude...
WE are not computing Turing machine as brain as said Roger Penrose and his friend
Anirban Bandyopadhyay which is way more advanced on A.I. new architecture than anyone else... I read his book "Nanobrain" 4 years ago...It is way over LLM...It is the next level in decades to come..It is a non Turing architecture coupled to a new mathematics.Anirban is the genius who described the electro-magnetic properties of microtubules...
We are as said Steiner : body/soul/spirit rooted naturally in the living cosmos in a way any A.I. could never be because it does not live, does not have an individual soul connected to the Spirit.....It is a hive mind... This Hive mind already exist in the cosmos as described by three thinkers who did not know each other : Rudolf Steiner in 1920, John C. Lilly in 1970 in his autobiography , and Michael Kerner in 2007 in 2 books..
I stop here... I suggested some names and books to read for those who want to understand over what is commonly found ...