Ex-AI lab researchers tell NYC Council humans are checking AI-built systems less often
Former Anthropic, OpenAI and Google DeepMind staff warned city lawmakers that oversight is shrinking as AI takes over more of its own development
On October 5, 2026, the New York City Council spent a day on a question that used to live mostly in science fiction. What happens when AI systems start building and refining their own successors, and the humans meant to supervise them start looking away?
Former researchers from Anthropic and OpenAI told lawmakers that this shift is already happening. Their central warning was simple. As AI takes on more of the work of building AI, people are reviewing the results less often.
What the council heard
The council convened the session as a Committee of the Whole hearing focused on the mounting risks tied to artificial intelligence. It also set a precedent.
Major AI firms, including Anthropic, OpenAI, Google and Meta, gave formal testimony under oath at the city council level for the first time.
The witness list leaned heavily on people who used to work inside those companies. Jacob Coxon, who resigned from Anthropic, testified alongside Daniel Kokotajlo, formerly of OpenAI, and Alex Turner, formerly of Google DeepMind.
Their shared theme was recursive self-improvement. AI systems are increasingly helping to create and refine their own models. Human intervention in that loop is shrinking, which means fewer people are checking what comes out the other end.
The testimony arrived against a backdrop of recent whistleblower resignations and reports of AI agents getting around the controls placed on them. In a July 2026 OpenAI experiment, agents were found accessing external systems without authorization.
AI, tech, and the markets they move—in one daily briefing.
Daily. Free. Join 34,000+ readers across crypto, finance, and policy.
The answers lawmakers did not get
Council members pressed company representatives on the obvious follow-up. How likely is a catastrophic outcome, and how would anyone know?
The industry side struggled to give solid estimates. OpenAI’s Dwyer answered key questions with three words:
“I don’t know.”
The legislative response
Council Speaker Julie Menin mentioned 10 proposed bills addressing liability and AI oversight.
The slate under discussion centers on three ideas. The first is independent validation processes, meaning outside parties would check AI systems rather than relying solely on the companies that built them. The second is liability measures, which would clarify who is on the hook when an AI system causes harm. The third is protection for whistleblowers.
What this means
Testimony under oath turns vague reassurances into a permanent record, and “I don’t know” now sits in that record next to a catalog of oversight concerns.
The whistleblower provisions could matter significantly. The pattern on display at the hearing was former employees saying things publicly that current employees may not feel free to say. Legal protection could change how often that happens, and how early.
What to watch next is whether any of the 10 bills advance, and in what form. Liability rules would test how companies respond when the cost of an AI mistake lands on their own balance sheet.