Skip to content
Advertisement
Epoch AI tests six leading LLMs, and none match its own employees

Epoch AI tests six leading LLMs, and none match its own employees

Photo: Merlin Lightpainting / Pexels

The research group's new automation reports found top models averaging approximately 65% of human-level work quality on real internal tasks

Epoch AI put six of the most capable large language models through a job interview of sorts. None of them got the job.

The research organization’s new Epoch Automation Reports, released on October 8, 2026, found that no tested model met the quality bar Epoch sets for work done by its own human employees. The best performers landed at roughly two-thirds of the way there.

How the test worked

Epoch’s evaluation used open-ended tasks drawn from real work inside the organization, including graphic design and research design. Human graders then scored the output against Epoch’s internal standards. The goal was to measure something closer to actual job performance than to a trivia contest.

The scoring scale was anchored to people. A score of 100% represents work of the quality a human employee at Epoch would be expected to produce.

Advertisement

The scoreboard

Two models shared the top spot. Claude Fable 5.1 and GPT-6 Astra each averaged approximately 65% across the tasks.

At the bottom sat Gemini 3.8 Flash, which scored 42%. That leaves a gap of more than 20 percentage points between the leaders and the trailing model.

No model crossed the 100% line. Not one reached parity with the humans whose work set the standard.

The models did better on structured work, with coding and data analysis standing out as relative strengths. Every model struggled with open-ended tasks that called for judgment or required tailoring work to a particular audience.

The leaderboard has also held steady. As of November 17, 2026, the rankings had not changed since the October release.

Why Epoch built another benchmark

Epoch’s automation reports are designed to address shortcomings in conventional performance measurement by reflecting the tasks people actually perform at work. The project sits within Epoch’s broader mission of tracking AI progress and capabilities accurately.

What this means

Models that look superhuman on structured tests landed well short of human quality once the work required judgment calls. Structured work like coding and data analysis appears to be where current models pull their weight. Tasks that hinge on audience awareness and discretion still seem to need a person in the loop, or at least a person checking the output.

Two rival labs sharing the top score suggests no single developer has pulled decisively ahead on this kind of open-ended work. Meanwhile, the 42% result for Gemini 3.8 Flash illustrates how wide the spread can be among models marketed for serious use.

Between October 8 and November 17, 2026, no model moved up the rankings. The next model release will face a clear and public test: whether it can push past approximately 65% on work that real employees do every day.

Disclosure: This article was edited by Diego Almada Lopez. For more information on how we create and review content, see our Editorial Policy.