Top AI executives from Anthropic, OpenAI, Meta, and Microsoft urge policymakers to regulate self-improving AI

More than 20 AI executives and researchers are urging governments to prepare for the possibility that artificial intelligence begins accelerating its own development, potentially compressing years of progress into months.

The group includes Anthropic co-founder Jack Clark, OpenAI Chief Scientist Jakub Pachocki, Geoffrey Hinton and Yoshua Bengio, alongside researchers and executives from Meta and Microsoft.

In a paper published Monday, the authors examine whether increasingly automated AI research and development could trigger what they call an “intelligence explosion,” where better AI systems help build even more capable successors at an increasingly rapid pace.

The researchers stressed that such an outcome remains uncertain. Constraints including computing capacity, long training runs and difficulties automating parts of AI research could slow or prevent the process.

Still, they argue that early evidence is strong enough for policymakers to begin preparing now. The paper calls for greater visibility into how much AI research is being automated inside leading labs, independent evaluation and mechanisms that could slow or constrain development if necessary.

The concern is becoming less theoretical as AI systems take on a growing share of their own development.

Anthropic said earlier this month that Claude now leads about 26% of its AI research and development work under human supervision, up from almost none at the beginning of the year. The company said Claude also collaborates with employees on more than 90% of its R&D work.

The authors warn that more capable systems could eventually reduce human involvement in AI development, potentially making it harder to oversee increasingly powerful models.

They also recommend that governments prepare emergency response plans for different scenarios and consider stronger oversight of the data centers used for highly automated AI research.

The paper comes amid broader calls from leading AI developers for governments and companies to put stronger safeguards around frontier systems as their capabilities and autonomy continue to increase.