Decision Index puts new decision models to the test on speed and cost

Artificial Analysis, the independent AI benchmarking firm best known for its Intelligence Index, now has a scorecard for a newer kind of AI system. The Artificial Analysis Decision Index rates decision models on three things: accuracy, speed and cost.

The early verdict is a familiar trade-off. Decision models are faster and cheaper, but they score worse on knowledge and reasoning tasks.

What the Decision Index measures

It tracks three dimensions at once. The first is accuracy, meaning how often a model gets the answer right. The second is speed, meaning how quickly the answer arrives. The third is cost, meaning how much each answer costs to produce.

Viewed together, the results show a clear split. Decision models do well on speed and cost. They fall behind on tasks that need broad world knowledge or multi-step reasoning.

So what is a decision model, anyway

Decision models are a new category of AI system. They have also been called System One models or typed decision engines.

A traditional large language model can write an essay, debug code or draft a breakup text. A decision model does something much narrower. It picks from a set of options defined in advance, such as yes or no, or one answer from a multiple-choice list.

The output is probabilistic and structured. The model doesn’t ramble. It hands back a choice along with how confident it is.

That narrow focus suits high-volume work. Examples include routing a support ticket, flagging a transaction or sorting a document into a bucket.

The category took shape around mid-to-late September 2026. TypeSafe AI’s Jev launched decision models built around typed probabilistic outputs around September 15, 2026.

The numbers behind the hype

Median latency for these models is approximately 148 ms. Cost runs around $0.11 per 1,000 requests.

On comparable tasks, decision models can be 60 to 400 times cheaper than LLMs while maintaining high accuracy. The key qualifier is “comparable.” These are bounded tasks where the possible answers are known in advance.

By October 2026, the list of providers included Cloudflare’s Clef, Perplexity’s pplx-decider, and a preview of OpenAI’s Decisions API.

Why an index for this matters

Artificial Analysis built its reputation by putting AI models side by side on shared tests. Its Intelligence Index became a common reference for comparing large language models.

The benchmark results support an argument the category’s builders have been making. Decision models aren’t meant to replace LLMs. They are meant to take over the repetitive, bounded calls that LLMs are overqualified and overpriced for.

What this means for AI buyers and builders

For companies running AI at scale, the main takeaway is about routing. If a task has a fixed set of answers, sending it to a full LLM may mean paying a premium for capability you never use.

The Decision Index findings point toward a split architecture. Cheap, fast decision models could handle the high-volume yes-or-no calls. Larger LLMs could be reserved for the work that needs knowledge or reasoning.

The fact that OpenAI is previewing its own Decisions API suggests at least one major LLM provider would rather offer the cheaper option than lose that business to it.

The accuracy gap is the risk to watch. A model that is 400 times cheaper loses its appeal if it gets the wrong answer on a task that turns out to need reasoning.