Skip to content
Advertisement
UC Berkeley researchers launch FrontierCS 2 to test whether AI agents can beat published research

UC Berkeley researchers launch FrontierCS 2 to test whether AI agents can beat published research

The new benchmark gives AI agents five hours and no internet to improve on code from academic papers, and the best model passed 58% of the time

A team led by UC Berkeley researchers has released FrontierCS 2, a benchmark that asks a pointed question: can an AI agent outperform published computer science methods without reading the paper or searching the web?

The answer, for now, is “sometimes.” The top performer, Claude Opus 5.5, cleared 58% of tasks. Sixteen tasks went unsolved by every agent tested.

How FrontierCS 2 works

The paper landed on arXiv on October 21, 2026, under the identifier arXiv:2610.29932. The project is spearheaded by Berkeley PhD student Qiuyang Mang, with collaborators from institutions including Princeton.

Roughly 60 researchers contributed to the effort.

FrontierCS 2 contains 48 open-ended tasks spread across 13 domains. The list covers areas such as machine learning, databases, and robotics.

Advertisement

Each task starts with an existing code repository tied to an academic paper. The agent’s job is to rebuild and improve that code within a five-hour window. Success means beating the original implementation.

The “better” part is graded on several fronts. Agents are judged on working performance metrics plus hidden workloads they never see in advance.

The no-papers, no-web rule is the other key constraint. An agent can’t look up how the original authors solved the problem. It has to work it out from the code and its own reasoning.

The leaderboard so far

The initial evaluation ran 12 different agent-model pairs through the task set. Claude Opus 5.5 led the pack with a 58% pass rate.

The 16 tasks that nobody solved are arguably the more interesting number. That means at least one agent cracked 32 of the 48 tasks, while a full third of the benchmark stumped the entire field.

The researchers also introduced an updated capability index. It is designed to reflect the median difficulty of tasks relative to the abilities of the AI systems being tested.

Updates in November expanded the leaderboard with additional models. Claude Sonnet 5.5 joined with a 50% pass rate, eight points behind its larger sibling.

The same round of updates addressed concerns about reward hacking.

Why the design breaks from older benchmarks

FrontierCS 2 is an evolution of the original FrontierCS benchmark introduced during the ICML 2026 conference, which emphasized expert-curated algorithmic challenges. FrontierCS 2 marks a pivot towards real-world environments, allowing agents to inherit functioning codebases from actual academic research.

The hidden workloads are a key part of that design. Visible tests can be memorized or overfit. Tests the agent never sees force it to build something that genuinely generalizes.

The benchmark also has a built-in defense against going stale. More than 100 additional private tasks are planned for future release.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.