Someone Got a 744B AI Model Running on a 25GB Laptop. No GPU Required.

A developer just got a 744-billion-parameter AI model running on a laptop with 25 GB of RAM. No GPU. No cloud. Just a 1,300-line C program and a lot of patience.

This happened yesterday. Same week OpenAI launched GPT-5.6 to general availability — their most capable model yet, available to anyone willing to pay. The two events couldn't be more different in spirit. And they tell you everything about where AI is actually heading.

The 744B Model That Runs On Your Laptop

GLM-5.2 is Z.ai's flagship open-source model — 744 billion parameters, MIT-licensed, competitive with GPT-5.5 on agentic benchmarks. We've covered it before. It's a beast. Normally you'd need a multi-GPU server with 400+ GB of VRAM to run it.

A developer who goes by JustVugg on GitHub looked at that model and thought: what if I ran it on my laptop?

The result is Colibrì — a project that's racked up 700+ stars in its first day on Hacker News with 450 upvotes and 115 comments. And the engineering behind it is genuinely brilliant.

Here's the key insight: GLM-5.2 is a Mixture-of-Experts model. Out of those 744 billion parameters, only about 40 billion activate for any given token. And of those 40 billion, about 17 billion are "dense" — they're used on every single token. The other ~11 billion are "routed experts" — specialized sub-networks that only load when the model decides it needs them.

So JustVugg split the model in two:

The dense part — attention layers, shared experts, embeddings — stays permanently loaded in RAM. Quantized to 4-bit, that's about 9.9 GB. Always there. Always ready.

The 21,504 routed experts — 256 experts per layer across 75 layers, each about 19 MB at 4-bit — live on disk as a ~370 GB file. They're loaded on demand, cached with an LRU eviction strategy, and backed by the OS page cache as a second-level buffer.

The engine itself is a single C file. ~1,300 lines. No BLAS (those optimized linear algebra libraries that everyone uses). No Python at runtime. No GPU. Just integer dot-product kernels with AVX2 CPU instructions, doing int8 and int4 matrix math by hand.

It even implements native MTP speculative decoding — GLM-5.2's own multi-token prediction head drafts tokens ahead of time, and the main model verifies them in batch. When it works (and the draft head needs to be 8-bit, not 4-bit, for this), it generates 2.2 to 2.8 tokens per forward pass instead of one. That's a 2-3x speedup for free.

Oh, and it has a learning cache. The engine tracks which experts your actual usage patterns route to, and pins the hottest ones in spare RAM. The model literally gets faster the more you use it.

Let's Talk About the Speed

Here's where I have to be honest with you: this isn't fast.

On JustVugg's machine — a 12-core laptop with 25 GB of RAM and an NVMe SSD — cold decode runs at about 0.05 to 0.1 tokens per second. For context, a typical API model serves 50-100 tokens per second. So we're talking roughly 500 to 1,000 times slower.

To generate a single token, the engine has to read about 11 GB of data from disk (75 layers × 8 experts × ~19 MB each). When the cache is cold — meaning the experts you need aren't already in RAM — you're bottlenecked by your SSD's random read speed. Even a fast NVMe tops out around 1 GB/s in random reads. That's the physics.

When the cache warms up — either through repeated use (the learning cache kicks in) or by pinning hot experts — the situation improves dramatically. If the experts you need are already in RAM, the only bottleneck is CPU math, and 12 cores doing int8 dot products can actually move at a reasonable clip.

But the cold-start reality is brutal. A short response might take minutes. A long one? Go make coffee. Actually, go make coffee, drink it, and make another one.

And JustVugg is refreshingly honest about this. The README doesn't pretend. It says: "This is not fast. It is a 744B frontier-class model answering correctly on a machine that costs less than one H100 fan."

So Why Does This Matter?

If it's slow, who cares? You can just use the API, right?

Here's why this matters — three reasons.

Reason one: the architecture is a preview of what's coming. The streaming-experts approach isn't specific to GLM-5.2. Every major model coming out of China right now — DeepSeek V4, Qwen3.7-Max, Kimi K2.7 — uses Mixture-of-Experts. The same dense/routed split that Colibrì exploits applies to all of them. As SSDs get faster (PCIe Gen 5 already hits 14 GB/s sequential) and as quantization techniques improve, the "stream experts from disk" strategy becomes increasingly viable on consumer hardware.

Two years ago, running a 70B model locally was aspirational. Today, a 9B model can beat 671B models at math. And now a 744B model runs — slowly, painfully, but runs — on a laptop. The trajectory is clear.

Reason two: it changes the interface question. The top comment on Hacker News makes a point that stuck with me. User walrus01 noted that even at 1 token per second, a locally-hosted model can be "very useful if you give it a project to do something overnight and metaphorically walk away from it."

This is the overnight model paradigm. You don't chat with a 744B model running on your laptop. You give it a job. A batch task. A research deep-dive. A code refactoring project. Something that would take a human hours anyway. You queue it up before bed, and the model grinds through it while you sleep.

At 0.1 tok/s, you get about 360 tokens per hour. Over an 8-hour night, that's roughly 2,880 tokens — about 2,000 words of output. That's a detailed report, a comprehensive code review, a multi-file refactor plan. Not blazing fast, but genuinely useful for the right tasks.

And this is where the interface matters. Chat is the wrong metaphor for slow local models. A ticket system — submit a task, get notified when it's done — fits perfectly. This is exactly how CopperRiver works: you give it a job, it works through it, and you don't have to sit there watching tokens stream.

Reason three: sovereignty. GLM-5.2 is MIT-licensed. When you run it locally, nobody can take it away from you, rate-limit you, change the pricing, deprecate it, or monitor your usage. Two weeks ago, the US government decided who gets access to frontier models like GPT-5.6 Sol and Claude Fable 5. Open-source models aren't subject to that. A 744B model running on your laptop is the ultimate "nobody can shut this off" scenario.

The Bigger Picture

The same week Colibrì launched, OpenAI released GPT-5.6 to general availability. It's a remarkable model — state-of-the-art on coding, knowledge work, cybersecurity, and science benchmarks. It introduces "ultra" mode, which coordinates four parallel agents to tackle complex tasks. On the Artificial Analysis Coding Agent Index, GPT-5.6 Sol scores 80 — beating Claude Fable 5 by 2.8 points while using less than half the tokens.

But there's a detail in OpenAI's developer guide that made me laugh. They explicitly tell developers: "Avoid generic brevity instructions." Apparently GPT-5.6 is "more sensitive" to prompts like "be concise" or "keep it short." In internal evaluations, replacing long system prompts with minimal ones improved scores by 10-15% while cutting cost by 33-67%.

So the most advanced model from the most well-funded AI company on Earth can't handle being told to "be brief." Make of that what you will.

Here's the tension: on one side, you have OpenAI spending billions on compute, shipping genuinely impressive models, and slowly building a moat around their ecosystem. On the other side, you have a solo developer with a laptop and 1,300 lines of C running a model that's in the same ballpark of capability — not speed, not polish, but raw intelligence — without paying OpenAI a dime.

Both approaches are valid. Most people should just use the API. It's faster, cheaper for most use cases, and frankly more reliable. But the Colibrì project proves something that the API vendors don't want you to think about too hard: the models are commoditizing. The intelligence is leaking out of the cloud and onto the desktop.

What Actually Runs Locally Today (Without Losing Your Mind)

Look, Colibrì is a proof of concept, not a productivity tool. If you want to actually use AI locally today — at speeds that don't make you question your life choices — there are better options.

Smaller open-source models are the sweet spot for local deployment. Models like Qwen3.6 (35B, MoE with only 3B active parameters) run at genuinely usable speeds on a Mac with 32 GB of unified memory. We're talking 20-40 tokens per second — fast enough for real coding assistance, not just overnight batch jobs.

The math is simple: MoE models punch above their weight. A 35B MoE model with 3B active parameters runs as fast as a 3B dense model but performs like something much larger. That's the same architectural insight that makes Colibrì possible — just applied at a scale that's practical for daily use.

This is why CopperRiver runs on open-source models by default. You get the speed of local execution, the privacy of data that never leaves your machine, and the cost profile of "free" (or close to it). You don't need to run GLM-5.2 at 0.1 tok/s to get value from local AI. You need the right model for your hardware, running tasks that match its strengths.

The Takeaway

Colibrì is the kind of project that makes me optimistic about AI. Not because it's practical — it isn't, at least not yet. But because it proves that the gap between "elite cloud model" and "what you can do at home" is closing in ways that the big labs can't control.

A solo developer, working alone, wrote 1,300 lines of C and made a 744-billion-parameter model answer questions on a laptop. That's not supposed to be possible. The model is supposed to live in a data center with armed guards and a $20/month subscription.

Except it doesn't have to.

If you want to see what practical local AI looks like — models running at usable speeds, automating real tasks on your Mac — check out CopperRiver. It's not 0.1 tok/s. It's a desktop AI assistant that actually gets things done.