DeepSeek V4.1 Flash hits 848 tokens per second on Nvidia’s GB300 NVL72

DeepSeek V4.1 Flash now generates 848 tokens per second per agent on Nvidia’s GB300 NVL72 system. A month ago, that number was 596.

That is an extra 252 tokens per second without swapping out the hardware.

What changed in a month

The figure that matters here is per-agent decode speed. Think of it as how fast a single AI session can type out its answer once it starts responding.

Going from 596 to 848 tokens per second on the same GB300 NVL72 setup points to gains in the software layer. That covers serving stacks, kernels, and model-level tweaks, not new silicon.

The research findings frame this as a joint effort across model design, serving software, and hardware. No single breakthrough did the heavy lifting.

There are bigger numbers floating around too. In specialized configurations, aggregate throughput reportedly topped 1 million tokens per second per GPU.

The reported cost in those setups is about $0.002 per million tokens. That estimate applies under large-hyperscaler pricing scenarios, so your average startup should not expect that invoice.

Inside the model and the machine

DeepSeek V4.1 Flash is a multimodal Mixture-of-Experts model with 552 billion total parameters. It launched around September 10, 2026.

In this model, only 8 billion parameters are active during prefill, the stage where it reads your prompt. During decoding, when it writes the response, that rises to 16 billion active parameters.

The model uses a causal encoder-decoder architecture. It handles both text and image inputs and supports a context window of 1 million tokens.

DeepSeek compresses the KV cache down to roughly 890 bytes per token. Smaller memory per token means more sessions can share the same hardware before things get crowded.

On the hardware side, the GB300 NVL72 packs 72 GPUs into a single rack. The findings credit it with improvements over its predecessor, the GB200 NVL72, particularly for workloads with high concurrency and long contexts.

A model that keeps getting updates

DeepSeek has not treated V4.1 Flash as a finished product. The model received a version update labeled 1017 in mid-to-late October 2026.

The findings describe DeepSeek-AI’s strategy as focused on fast inference. The 848 tokens per second figure is a fairly direct expression of that priority.

What this means

For companies building AI products, the headline number is less important than the trajectory. If per-agent speed can climb by 252 tokens per second in a month on fixed hardware, capacity planning gets a lot harder to pin down.

Per-agent speed and aggregate throughput also measure different things. One tracks how fast a single session responds, while the other tracks total output across the system, so they should not be read as interchangeable.

There are caveats worth keeping in view. The most eye-catching throughput and cost numbers come from specialized configurations and large-hyperscaler pricing, not typical deployments.