A 20B AI Model Trained in Ternary Just Hit 218 Tokens/Second on a $599 Mac Mini

For the last two years, the recipe for running AI models on your own hardware has been the same. Take a big model trained at full precision. Squeeze it. Shave bits off until it fits in your RAM. Accept that it got dumber. Run it.

Quantization. 8-bit, 4-bit, 2-bit. GGUF, AWQ, GPTQ. Every few months, someone finds a cleverer way to compress a model that was designed to run at 16 bits into something your laptop can handle. The results keep getting better. But the approach never changes: build a high-precision model, then degrade it.

DeepGrove just took a completely different path.

Yesterday, they released Maple-Preview: a 20-billion-parameter AI model that was never trained at full precision. It learned in ternary — every weight in the model is either -1, 0, or +1. Not "quantized to ternary after training." Ternary from birth. And the results are kind of absurd.

It runs at 218 tokens per second on a $599 Mac mini. That's the base model M4 with 16 GB of RAM — the cheapest Mac Apple sells. It runs at 127 tokens per second on an iPhone. It solves International Mathematical Olympiad problems. The entire model is 5.31 GB on disk.

Let me put that in context. GPT-OSS-20B — OpenAI's own open-weight 20B model — runs at 45.9 tokens per second on the same Mac mini, eats 11.6 GB of RAM, and its checkpoint is more than twice the size. Maple is 4.8× faster and less than half the footprint. Gemma 4 12B, a model half Maple's size, crawls at 9.1 tokens per second on the same hardware. Maple is 24× faster than a model it outweighs.

This isn't an incremental improvement. It's a different category entirely.

What Ternary Actually Means

Standard neural network weights are 16-bit floating point numbers. Each weight can be any of 65,536 values. A 20B parameter model at 16-bit precision needs roughly 40 GB of memory just to store its weights.

Quantization compresses those 16-bit values into fewer bits. 4-bit quantization maps them to 16 possible values. You lose some accuracy, but the model shrinks to a quarter of its size. It works well enough that 4-bit is now the standard for local inference.

Ternary goes further than any practical quantization scheme. Each weight has exactly three possible values: -1, 0, or +1. That's 1.58 bits per weight. Four weights pack into a single byte. A 20B parameter model in ternary needs about 3.9 GB for the raw weight data — and Maple's total 5.31 GB includes scales, biases, embeddings, and KV cache infrastructure. Everything.

Here's why that matters for speed. At batch size 1 — which is what you're doing when you chat with a model on your phone — generating each token is mostly a sequence of matrix-vector multiplications. The model reads its weights from memory, multiplies them by the current activation vector, produces the next layer's input. Performance is bottlenecked by memory bandwidth, not raw compute.

With ternary weights, matrix multiplication basically becomes addition. Multiplying a vector by -1, 0, or +1 doesn't require a multiply instruction — you negate, zero, or pass through. DeepGrove's custom kernels pack four ternary weights per byte, use masked-multiply loops to compute dot products without ever unpacking them, and process 512 input values per iteration on a 32-lane SIMD group.

The math is clean: Maple reads approximately 317 MB of data per token at the benchmark's 640-token context length. The M4 Mac mini has 120 GB/s of peak DRAM bandwidth. That's 2.64 ms per token, or 379 tokens per second at the theoretical bandwidth limit. Their current 218 tok/s is using about 58% of peak bandwidth. There's headroom left.

Why Everyone Else Quantizes After Training

Every other "small model on your device" approach I've covered follows the same pattern. Train a model in full precision. Then compress it.

TurboFieldfare streams expert weights from your SSD so a 26B model fits in 2 GB of RAM. WASTE streams an entire 2.78-trillion-parameter model from NVMe at half a token per second. PrismML compressed a 27B model into 3.9 GB and ran it on an iPhone. Colibri hand-wrote 1,300 lines of C to run a 744B model on a laptop with no GPU. DeepGrove's own earlier model, Bonsai, was ternary — but only 0.5 billion parameters. A proof of concept.

All of these are impressive engineering. But they share a fundamental limitation: the model was designed to work at high precision, and compression is a compromise. You lose 5-15% of the model's intelligence every time you quantize. The more aggressively you compress, the more you lose. It's a tax on every inference.

DeepGrove's argument is that this is backwards. If you know the model is going to run at ternary precision, train it that way. Let the model learn to be smart with just three values per weight. Don't build a high-precision brain and then lobotomize it. Build a brain that was always meant to work this way.

In their words: "We believe that the precision a model runs at should be the precision it learns at."

This is the part that should actually unsettle the quantization ecosystem. DeepGrove isn't saying quantization is useless. They're saying the entire pipeline — train big, compress later — leaves performance on the table at both ends. A compressed model can never be as efficient as a natively-ternary one because its architecture was optimized for a different precision. And it can never be as smart as it could be because compression degrades it. You're paying a tax at training time for a precision you'll never use, then paying again at inference time to approximate what you lost.

The comparison table on their model card makes this visceral. Ternary Bonsai 27B — which is a Qwen3.6 27B model converted to ternary after training — runs at 13.1 tokens per second on the Mac mini. Maple-Preview, which is natively ternary at 20B, runs at 169.6 tokens per second. Same precision format. 13× faster. Because the model was designed for it from the first gradient.

The Benchmarks: Actually Smart, Not Just Fast

Speed means nothing if the model is dumb. So here are the numbers.

Maple-Preview averages 78.7% across four reasoning benchmarks:

• LCBv6 (LiveCodeBench v6): 75.1% — competitive programming
• AIME 2026: 87.5% — elite high-school math olympiad
• HMMT 2026: 78.8% — Harvard-MIT Math Tournament
• GPQA-D: 73.5% — graduate-level science questions

For comparison, Qwen3.5 35B-A3B — a model nearly twice Maple's total size — scores 74.6% on LCBv6. Maple beats it. A 20B ternary model outscoring a 35B full-precision model on competitive coding, while running 5-16× faster.

The model also solves IMO 2024 Problem 1 cleanly — full 7/7 marks. The International Mathematical Olympiad is the hardest high-school math competition on the planet. A perfect score on any single problem is genuinely difficult for most adult humans. The model generated its proof on a MacBook Pro at 281.5 tokens per second.

Now, the honest caveats — and DeepGrove is upfront about all of them. This is a "preview" release. It received minimal post-training for agentic tasks and only small-scale general reinforcement learning. It's great at math and reasoning, but if you ask it to use tools, follow complex multi-step instructions, or drive an autonomous workflow, it's probably not there yet. They explicitly say it "may underperform on agentic benchmarks."

So this is not a model you'd use to run an AI agent that browses the web and executes terminal commands. Not yet. It's a reasoning engine. But it's a reasoning engine that runs in your pocket, costs nothing per query, never sends your data anywhere, and is MIT-licensed.

The Architecture: Designed for the Hardware

Maple-Preview is a Mixture of Experts model. 20.2 billion total parameters, but only 1.49 billion active per token. Eight of 256 experts fire on each forward pass. That small active footprint is why it's so fast — each token requires very little actual compute, and the bottleneck becomes how quickly you can stream the relevant expert weights from memory.

The architecture choices were made with a specific piece of hardware on the bench: a Mac mini. DeepGrove tested every configuration by measuring real inference speed on the M4, not theoretical FLOPS. They started with a 30-layer, 224-expert design and optimized down to 24 layers with 256 experts as a deliberate compromise between model quality and inference efficiency.

Attention is hybrid: 6 layers of full global attention and 18 layers of sliding-window attention capped at 512 tokens. This bounds KV cache growth so the model doesn't balloon in memory at long context. It supports 131,072 tokens of context — and even at that length, the projected memory stays well within a 16 GB budget.

The custom MLX kernels are where the real engineering lives. DeepGrove wrote ternary matrix-vector multiplication kernels that pack four weights per byte and compute dot products through masked-multiply accumulation — essentially reading the packed byte, masking individual 2-bit codes, scaling activations to match bit positions, and accumulating in FP32. No unpacking into a temporary matrix. No wasted memory bandwidth.

They also built "FlashHead" — an approximate output head for the most expensive part of decode: projecting from the model's hidden state to a vocabulary of ~151,000 tokens. FlashHead clusters the vocabulary into 4,748 groups, quickly identifies the top-512 relevant clusters using 4-bit centroid scoring, and only computes full-precision projections for those. This turns an O(hidden × vocab) operation into something dramatically cheaper, with negligible quality loss.

All of this runs on Apple Silicon through MLX, Apple's open-source machine learning framework. There's also a separate CUDA/Triton implementation for GPU servers. The whole model, all custom kernels, the inference engine — open source under MIT.

What This Means for Local AI

I've been tracking the local AI movement closely. The trajectory is unmistakable: every month, someone finds a way to do more with less hardware. SSD streaming, NVMe offloading, aggressive quantization, mixture-of-experts architectures that keep active parameter counts tiny. The gap between "models you can run at home" and "models locked behind a corporate API" keeps shrinking.

Maple-Preview suggests that gap might close faster than anyone expected — and not just through increasingly desperate compression tricks. If natively low-precision training works at scale, it changes the math on everything. Models become smaller by design, not by degradation. Inference becomes faster because the fundamental operations are simpler — addition instead of multiplication. The economics of on-device AI shift from "can we squeeze this cloud model onto a phone?" to "why would you ever use a cloud model for this?"

DeepGrove puts it well: "We find it easy to imagine a world where most inference is done on smaller, personalized models and infrequent, exceedingly difficult tasks are offloaded to cloud models."

That world looks something like this: your phone runs a fast, private, always-available model for 95% of what you need — drafting, reasoning, math, quick lookups, conversation. When you need something bigger — complex multi-file code generation, deep research across dozens of sources, specialized professional tasks — it reaches out to a cloud model. The local model handles the mundane. The cloud model handles the exceptional. You stop paying API taxes on every interaction.

We're not there yet. Maple-Preview can't use tools. It's not agentic. It's a preview, as the name suggests. But the speed-to-quality ratio is so dramatically better than anything else in its weight class that it's hard not to see this as a genuine glimpse of where consumer AI is heading. Not cloud APIs at $15 per million tokens. A 5 GB model that lives on your device, costs nothing to run, and solves olympiad math faster than you can read the answer.

The full model — Maple, not Maple-Preview — is coming later. DeepGrove says they're continuing training to improve general performance and agentic capabilities before the full release. If ternary-native training scales the way this preview suggests, the implications are significant. Not just for hobbyists running models on their laptops, but for the entire economics of AI inference.

In the meantime, you can try Maple-Preview in your browser, download the weights from Hugging Face, or run it locally on any Mac with MLX support. MIT licensed. No strings attached.

If you want to run open-source AI models locally — on your own hardware, without API fees, without your data touching someone else's server — CopperRiver handles the infrastructure so you can just use the model. It's a desktop AI assistant for Mac that connects to open-source models and lets you get actual work done. Plans start at $9/month. The local AI revolution was already underway. It just got 13× faster.