More than 20 leading AI researchers warn that automated AI research poses extreme risks
In case you didn't have enough, here's another AI warning: More than 20 AI researchers, including Geoffrey Hinton, Yoshua Bengio, and OpenAI research lead Jakub Pachocki, warn of a possible "intelligence explosion" driven by self-improving AI in a new research paper . "Although there remains much uncertainty, AI R&D automation might soon trigger one," they write.
According to the study , AI systems already write most of the code at the companies building them and could automate the entire AI R&D pipeline within years. Progress that normally takes years could then happen in months. The authors warn society might not keep up, control over superhuman AI could slip away, and power balances between nations, companies, and governments could erode. They urge policymakers to gain far more visibility into how AI research is being automated.
The paper joins a growing list of warnings. 42 leading mathematicians recently called for more attention to existential AI risks, several AI lab employees have voiced fears that AI could destroy humanity, and Pachocki himself said no lab has solved alignment well enough "to continue responsibly scaling at maximum speed for much longer." Some Anthropic employees are reportedly already scouting safe havens in case AI goes sideways . Ad Ad
AI News Without the Hype – Curated by Humans
Subscribe to THE DECODER for ad-free reading, a weekly AI newsletter, our exclusive "AI Radar" frontier report six times a year, full archive access, and access to our comment section.