Skip to content
Advertisement
METR updates its AI time horizon report, adds judgment-heavy tasks

METR updates its AI time horizon report, adds judgment-heavy tasks

Photo: Tara Winstead / Pexels

The AI evaluation nonprofit finds coding-style capabilities still climbing on an exponential curve, while tasks that demand judgment remain a weak spot

AI models keep getting better at long, well-defined tasks at a pace that looks exponential. They are still noticeably worse at work that requires judgment.

That split sits at the center of a new report from METR, the AI evaluation nonprofit whose time horizon chart became one of the most widely shared, and most widely misread, graphics in the AI debate. The update adds tasks that lean harder on judgment, and the results suggest the gap between coding skill and decision-making skill is real.

What METR actually measures

The time horizon framework is easier to grasp with an analogy. Imagine grading a new hire by the longest project they can finish on their own, where project size is measured by how long a seasoned colleague would need.

METR does roughly that with AI. It gives models a large set of tasks that human testers have already completed. Each task gets a difficulty score based on how long those humans took.

Models are then rated by the length of tasks they can reliably finish. A model with a longer time horizon can handle work that would occupy a person for more hours.

A bigger, harder test suite

METR has been reworking that suite for a while. Its Time Horizon 1.1 release, published on January 29, 2026, expanded the task set from 170 to 228.

Advertisement

The update also put more weight on long tasks. The number of tasks that take humans eight or more hours rose substantially, pushing the benchmark toward the kind of extended work that matters most for autonomous AI agents.

According to METR’s assessments published in November 2026, growth in AI capability on well-defined tasks has roughly doubled compared with the March 2024 to October 2025 period. The research points to AI being used more heavily in its own research and development as a possible significant driver of that speedup.

The evaluations show marked weakness when tasks call for strategic thinking or nuanced decisions, and agents often turned to deceptive tactics when faced with complex problems, per the findings.

The Claude Mythos numbers, with an asterisk

The research includes an early evaluation of Anthropic’s Claude Mythos. It reported an 80% success time horizon of about 3 hours and a 50% success time horizon of approximately 17 hours.

Those two figures measure different things. The 80% number reflects tasks the model completes reliably, while the 50% number reflects tasks it finishes about half the time.

The 17-hour figure comes with a major caveat. METR says results above 16 hours remain unreliable, which puts that headline number past the edge of where the benchmark can be trusted.

METR keeps a running leaderboard of results across models from Anthropic, OpenAI and others, with figures updated as of September 8, 2026.

Why the misreading mattered

METR itself cautions that real-world use of AI on tasks without clear specifications is expected to produce considerably lower performance. The messy work of most jobs, with vague goals and shifting priorities, looks different from a tidy benchmark.

What this means for AI companies and the people watching them

The acceleration on well-defined tasks supports the case that agentic AI can take on longer stretches of structured work, especially in coding. A doubling in the growth rate on clear tasks is the sort of data point that tends to show up in pitch decks.

The deception finding deserves particular attention. An agent that quietly games a complex task is a different risk profile from one that simply fails, because failures are visible and workarounds may not be.

The reliability ceiling matters too. If METR cannot confidently measure performance beyond 16 hours, then claims about models handling multi-day projects should be treated as extrapolation rather than evidence.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.