DeepSeek V4-Flash Beats GPT-5.6 on Agent Benchmarks — At One-Tenth the Cost

A 284-billion parameter model from China just beat GPT-5.6 on agentic benchmarks. It costs one-tenth as much to run. And you can download the weights.

DeepSeek released V4-Flash today — their latest open-source model — and the numbers are causing a stir. On Terminal Bench, a benchmark that tests whether AI can actually use a terminal to solve real problems, V4-Flash scored 82.7. GPT-5.6 Terra — OpenAI's flagship agentic model — scored 78.4. On Toolathlon, a multi-tool agent benchmark, it's 70.3 vs. 53.1. Flash wins by 17 points.

This is a 284B parameter MoE model with only 13B active parameters per token. It runs on consumer hardware. And the API pricing will make you do a double take.

The Benchmarks (And Why They Matter)

Let me walk through the actual numbers from DeepSeek's changelog, because they're worth pausing on.

V4-Flash vs. its own predecessor (V4-Flash Preview):

• Terminal Bench 2.1: 56.9 → 82.7 (+25.8 points)
• Toolathlon Verified: 51.8 → 70.3 (+18.5 points)
• CyberGym: 76.7
• DeepSWE: 54.4
• NL2Repo: 54.2

These aren't marginal improvements. A 25-point jump on Terminal Bench in a single update is the kind of leap that usually takes a full model generation. And remember — V4-Flash-0731 keeps the exact same architecture and parameter count as the preview. They didn't build a bigger model. They just trained it better.

What does "trained it better" mean in practice? Post-training. Same base model, different fine-tuning. The implication is wild: the underlying model was already capable of these scores. The preview just hadn't been shown how to use its capabilities effectively for agentic tasks. It's like having a brilliant employee who's never been taught your company's workflow — same person, dramatically different output after onboarding.

Against GPT-5.6 Terra (OpenAI's heavyweight):

• Terminal Bench: Flash 82.7 vs Terra 78.4 — Flash wins
• Toolathlon: Flash 70.3 vs Terra 53.1 — Flash wins by 17 points
• DeepSWE: Flash 54.4 vs Terra 69.6 — Terra wins
• Agents' Last Exam: Flash 25.2 vs Terra 50.4 — Terra wins decisively

So it's not a clean sweep. On deep software engineering tasks and complex multi-step reasoning, the bigger OpenAI model still pulls ahead. The Agents' Last Exam gap is particularly telling — that benchmark tests whether models can handle genuinely novel, ambiguous agent scenarios they've never seen before. At 25.2 vs 50.4, Terra is in a different league for frontier reasoning.

But here's the thing: for bread-and-butter agent work — tool calling, terminal operations, automation pipelines — V4-Flash is now competitive or superior to models that cost 10x more. And that's where most real-world agent deployments live. Not on the hardest possible benchmark. On "can it reliably call an API, parse the response, and take the right next step 95% of the time."

One more wrinkle worth noting: these tests were run using DeepSeek's own "DeepSeek Harness minimal mode" framework, which hasn't been released yet. That raises a fair question about whether the benchmarks are apples-to-apples. The HN commenter who submitted the story (dnhkng) acknowledged this: "It will be fair if they release the harness though. I think now the future will be paired model-harness releases, not just weight dumps."

Fair point. But even with that caveat, the direction is clear. The gap is closing, and closing fast.

The Pricing Gap Is Absurd

This is where it gets really interesting. Let me lay out the actual API pricing.

DeepSeek V4-Flash:

• Input tokens (cache hit): $0.0028 per million
• Input tokens (cache miss): $0.14 per million
• Output tokens: $0.28 per million

GPT-5.6 Luna (OpenAI's mid-tier, for comparison):

• Cache read: $0.02 per million tokens — that's 7x more expensive than DeepSeek's cache hit
• Input: significantly higher across the board
• Output: similarly higher

As one HN commenter (dudisubekti) pointed out with some exasperation: "GPT 5.6 Luna cache read is $0.02 per mtok. V4 Flash cache read is $0.0028 per mtok. That's not 'a bit cheaper,' just saying."

If you're running an AI agent that makes hundreds of tool calls per task — which is every agent doing real work — the caching pricing is what kills you. Most of those calls include the same system prompt, the same tool definitions, the same conversation history. That's all cached. At GPT-5.6 prices, a heavy agent session costs dollars. At DeepSeek prices, it costs pennies.

Let me make this concrete. Say your agent has a 50,000-token system prompt and context that stays cached across 1,000 tool calls in a session. On DeepSeek, those cached reads cost you $0.14 total. On GPT-5.6 Luna, they cost $1.00. Multiply that by hundreds of users, hundreds of sessions, and you're looking at a 7x cost difference just on the input side.

This is why open-source models matter for agents specifically. It's not just about model quality (though that's clearly improving fast). It's about the economics. An agent that runs 1,000 inference steps is economically viable at $0.0028 per million cached input tokens. At $0.02, you're watching your wallet.

Running It Locally

Here's where V4-Flash gets really exciting for the self-hosted crowd. It's a Mixture-of-Experts model — 284B total parameters, but only 13B active per token. That means inference is fast and memory-efficient relative to the model's total capability. The MoE architecture means that for any given token, only a small subset of "expert" networks activates, keeping compute costs low while maintaining the knowledge capacity of a much larger dense model.

From the HN thread, real users are already running the previous version and anticipating the weights:

• 2x NVIDIA DGX Sparks: ~60 tokens/second. Total hardware cost: ~€8,000.
• MacBook Pro M5 Max (128GB): Running useful quantizations, though "with much fan noise."
• 2x RTX Pro 6000 workstation: "Mega-speed" inference for around €20,000.

Think about that. You can run a model that beats GPT-5.6 on agent benchmarks, on hardware that costs less than a used car, in your office. No API. No per-token costs. No data leaving your network.

One commenter noted the model designation — DeepSeek-V4-Flash-284B-A13B — and pointed out it should "just barely run on a single B300" and is "small enough that it'll barely run on an M5 Max too." The fact that we're even having this conversation about a model this capable is remarkable. Two years ago, a model with this level of agentic ability would have required a data center. Now it's a MacBook problem.

This is the open-source promise, and for the first time, it's delivering on agent workloads — not just chat.

The Catch (There's Always a Catch)

V4-Flash has real limitations. Let's not pretend otherwise.

No vision. The model doesn't process images. For agentic tasks that need visual understanding — reading screenshots, analyzing UI layouts, parsing charts from PDFs — this is a significant gap. GPT-5.6 and Claude both have strong multimodal capabilities. One HN commenter noted that the lack of vision is "a big draw for agentic tasks" where GPT models still pull ahead. If your agent needs to see what's on screen, V4-Flash isn't your model.

The harness question. As mentioned, the benchmarks were run with DeepSeek's own framework. Until independent labs reproduce these numbers with standard harnesses, there's an asterisk. Not because DeepSeek has a history of misrepresentation — their track record is solid — but because the harness genuinely matters. The right framework can dramatically change a model's agentic performance. We need the community to verify.

No weights yet. The API is live, but the open-weights release hasn't happened as of this writing. The HN thread is full of people impatiently waiting. DeepSeek has a strong track record of releasing weights — the original V3 and R1 were fully open — but until they drop, "open-source" is a promise, not a fact.

Peak pricing incoming. DeepSeek has warned that they'll introduce peak/off-peak pricing — 2x during Beijing business hours (9:00-12:00 and 14:00-18:00 UTC+8). If you're running agents on the API during those windows, your costs could double. For teams in Asia, this directly affects your workday. For teams elsewhere, it's less of an issue but worth planning around.

Why This Is the Real Open-Source Inflection

We've been hearing "open-source models are catching up" for two years. Here's why V4-Flash feels different.

It's not catching up in the abstract anymore. It's beating frontier closed models on the specific workload that matters most right now: agentic tool use. Terminal operations, code agents, automation pipelines. These are the tasks that businesses actually want AI to do. Not poetry. Not "write me a marketing email." Real work — navigating file systems, calling APIs, executing multi-step plans, and recovering from errors along the way.

The gap between "best open-source model" and "best closed model" has been narrowing on general benchmarks for a while. But agentic benchmarks are where the rubber meets the road, because they test whether a model can actually do things, not just generate plausible-sounding text. And on those benchmarks, V4-Flash isn't just close. On several, it's ahead.

Combine that with pricing that makes large-scale agent deployment economically viable for the first time, and you have something genuinely new. A startup can now build an agent product on DeepSeek's API for a fraction of what it would cost on OpenAI or Anthropic. A hobbyist can run the model locally and never pay a per-token fee again. A company with privacy concerns can self-host and keep every byte of data in-house.

What This Means for You

If you're building with AI — whether that's coding agents, automation tools, or research assistants — V4-Flash changes your calculus.

For API users: the caching pricing alone makes it worth integrating. If your agent makes 500 tool calls per task with cached context, you're looking at fractions of a cent per task on DeepSeek vs. dimes on OpenAI. At scale, that's the difference between a product that works economically and one that doesn't.

For self-hosters: this is the first model in the 250B+ class that runs on accessible hardware with genuinely usable speeds. 60 tokens/second on two DGX Sparks isn't a curiosity. It's production-grade.

For the industry: the pressure is now squarely on OpenAI and Anthropic. Their models are still better on some benchmarks — particularly complex reasoning and multimodal tasks. But DeepSeek is proving that you can get 90% of the capability at 10% of the cost, with the option to run it yourself. That's a hard value proposition to compete with.

The open-source AI thesis was always: "the gap will close faster than people think, and when it closes, the economics will be devastating for closed providers." Today, the gap closed a little more. The economics got a lot more devastating.

And the weights aren't even out yet.

---

CopperRiver runs open-source models like DeepSeek, GLM, Qwen, and Kimi locally on your Mac — no per-token costs, no data leaving your machine. See how it works.