Kimi K3: The 2.8T Open-Source Model That Built a GPU Compiler and Designed a Chip

An AI model just designed a working computer chip. Not a schematic. Not a sketch. A real silicon design — timing verified at 100 MHz, 1.46 million standard cells, 0.277 MB of SRAM, packing an INT4 MAC array. Built in a single 48-hour autonomous session.

The model also wrote a GPU compiler from scratch. Not a toy — it built something called MiniTriton, with its own IR layer, optimization passes, and PTX code generation pipeline. On some workloads, it beats the actual Triton compiler that ships with PyTorch.

Oh, and it reproduced a complex computational astrophysics pipeline — reviewing 20+ papers, implementing 300+ equations of state, writing 3,000+ lines of Python, and spotting errors in published formulas. Work that would take an experienced researcher one to two weeks. It did it in two hours.

The model's name is Kimi K3, and it's the biggest open-source AI release of the year.

2.8 Trillion Parameters. Open Weights. What?

Kimi K3 dropped yesterday with 2.8 trillion parameters — making it the first open model to cross the 3T-class threshold. To put that in perspective: the previous generation, Kimi K2, was already pushing boundaries, and K3 claims a 2.5× improvement in scaling efficiency over it.

The architecture is where it gets interesting. They didn't just throw more parameters at the wall. Kimi introduced two new components:

• Kimi Delta Attention (KDA) — a rethinking of how attention information flows across sequence length
• Attention Residuals (AttnRes) — changes how information propagates across model depth

On top of that, they scaled up Mixture of Experts aggressively. K3 activates only 16 out of 896 experts per token, using what they call a "Stable LatentMoE" framework. That's the magic of MoE — you get the capacity of a 2.8T model but only pay the inference cost of activating ~16 experts worth of parameters at a time.

The model also has a 1 million token context window and native vision capabilities. It can look at screenshots and iterate on visual outputs — what Kimi calls "vision in the loop."

The Benchmark Numbers

According to Artificial Analysis, Kimi K3 lands at #4 out of 189 models on their Intelligence Index with a score of 57. That puts it ahead of virtually every open model and behind only the heaviest proprietary heavyweights — Claude Fable 5 and GPT 5.6 Sol.

But let's be honest about the trade-offs, because there are some real ones:

It's slower than average and more expensive than average — for a model whose main selling point is being open-source. And it's very verbose: during the Intelligence Index evaluation, it generated 130 million output tokens (the average model generates 63 million). That verbosity directly translates to higher costs at the API level.

Translation: Kimi K3 is brilliant, but it thinks out loud. A lot.

Simon Willison tested it with his now-legendary SVG pelican benchmark and noted the model burned through 16,658 output tokens — 13,241 of which were reasoning tokens — for what should be a simple drawing task. That cost 25 cents. For one pelican.

The Demonstrations Are Genuinely Insane

Let's talk about what actually matters here, because benchmarks are benchmarks and real-world demonstrations are real-world demonstrations.

The GPU compiler (MiniTriton): Kimi K3 built a compact Triton-like compiler from scratch. It has its own tile-level IR over MLIR, optimization passes, and a PTX code generation pipeline. On roofline benchmarks, it matches or beats both Triton and torch.compile. It sustained end-to-end nanoGPT training with stable convergence. A model built a compiler that rivals the one humans spent years optimizing.

The chip design: In 48 hours, K3 designed a chip — for serving a nano model built on its own architecture, no less. It used open-source EDA tools on the Nangate 45nm library, closed timing at 100 MHz within 4mm², and hit 8,700+ tokens/s decode throughput in simulation. A chip built by a model, for a model. If that sentence doesn't make you pause, read it again.

The research reproduction: It reproduced the I-Love-Q universal relations from computational astrophysics — cross-validating 20+ papers, implementing the full numerical pipeline, evaluating 300+ equations of state, finding inconsistencies in published formulas, and producing an interactive HTML dashboard. Two weeks of expert work. Two hours.

The 3D open-world game: It built a fully procedural browser-based 3D exploration game using Three.js WebGPU with forests, a log-cabin village, snowy mountains, and dynamic weather. Using vision in the loop to iterate between code and live screenshots.

I've been watching AI model releases for a while now, and most "look what our model can do" demos are carefully curated cherry-picks. But the breadth and autonomy here is different. K3 isn't being prompted step-by-step through these tasks. It's operating with minimal human oversight, sustaining long engineering sessions, navigating massive codebases, and orchestrating terminal tools on its own.

The Architecture Bet That Might Matter Most

Everyone focuses on parameter counts, but the architectural innovations in K3 are arguably the more interesting story for the long term.

Standard transformer attention has a scaling problem: as you increase sequence length and model depth, information degrades. KDA and AttnRes aren't just incremental tweaks — they're fundamental changes to how attention works across both sequence length (KDA) and model depth (AttnRes).

If these innovations are real and transferable, they could show up in other models. The open-source community will be dissecting this architecture the moment the full technical report and weights drop.

And the MoE scaling is notable too. 896 experts with only 16 active means the model has enormous capacity but maintains reasonable per-token compute costs. The "Stable LatentMoE" framework is their solution to the training instability problems that plague large MoE models. If it works as advertised, expect to see this pattern replicated.

The Elephant in the Room: Cost and Speed

Here's my honest take: Kimi K3 is a technical marvel that's currently impractical for the kind of high-volume, tool-calling work that real AI agents need to do.

At $3/$15 per million tokens (input/output), it's roughly 2× the industry average. When you factor in the verbosity — 130M tokens vs 63M average during evaluation — your actual spend per task is even higher. And at 62 tokens per second, it's not winning any speed contests.

For comparison, models like GLM-5.2 and DeepSeek V4 offer comparable bang-for-buck at significantly lower price points. The gap isn't enormous, but it matters when you're running an agent that makes 50+ tool calls per task.

The caching helps — cache hit pricing drops to $0.30/1M tokens, a 90% discount. But for workloads with high input diversity (which is most agentic work), caching only goes so far.

This is the classic open-source model dilemma: you can be the smartest model in the room, but if you're slow and expensive, people will reach for the cheaper option 80% of the time and only escalate to you for the hard problems.

What This Means for the Open-Source Race

Kimi has been pushing the open-model size frontier for nine of the last twelve months. They're not a flash in the pan — this is a sustained strategy. And with K3, they've drawn a clear line: open models can compete at the absolute frontier of capability.

The full model weights drop July 27, 2026. That's when it gets real. Once the weights are in the wild, the community will:

1. Optimize inference — vLLM, SGLang, and TensorRT-LLM teams will squeeze out the speed gap
2. Quantize aggressively — FP8, INT4, even lower. A 2.8T model at FP8 is ~280GB, which fits on a single 8×H200 node
3. Fine-tune for efficiency — reduce that verbosity problem and cut token costs
4. Study the architecture — KDA and AttnRes will get forked into other models

The speed and cost problems are solvable. The intelligence — that's the hard part, and K3 has it in spades.

The Bigger Picture

We're watching the open-source AI gap with proprietary models close in real time. A year ago, the idea that an open model could build a GPU compiler or design a silicon chip would have been laughable. Now it's a Tuesday.

The implications for AI agents are massive. Models like K3 can sustain the kind of long-horizon, autonomous work that real automation requires — navigating codebases, orchestrating tools, iterating on visual feedback, and maintaining context over millions of tokens. That's not a chatbot. That's a digital worker.

And when the weights are open, you can run it yourself. No API dependency. No per-token costs. No data leaving your machine. That's the promise that CopperRiver was built around — desktop AI agents powered by open-source models that actually do things. Kimi K3 is exactly the kind of model that makes that vision practical.

The frontier isn't just getting closer. For the first time, the open-source community is at the frontier. And the weights drop in ten days.

Set a calendar reminder. This one matters.