A Solo Dev Just Ran a 26B AI Model on a Base MacBook Air. The Math Is Absurd.

A 26-billion-parameter AI model should not run on an 8-gigabyte MacBook. The weights alone for Gemma 4 26B take up 14.3 GB of storage. Conventional wisdom says you need at least 16 GB of RAM just to load the thing, ideally 32. The smallest Mac that should handle it is an M-series Pro with unified memory to spare.

A developer named Drumi Hristov just made it run on the cheapest MacBook Apple sells. In 2 GB of RAM. At 5-6 tokens per second.

The project is called TurboFieldfare, it's open-source, it hit the top of Hacker News this week, and the engineering behind it is genuinely clever. Not "we used a smaller quantization" clever. "We rewrote how model loading works" clever.

The problem: MoE models are mostly dead weight

Gemma 4 26B-A4B is a Mixture of Experts model. That "A4B" means only about 3.88 billion parameters are active per token — the model routes each token through a small subset of "expert" subnetworks rather than using all 26 billion parameters every time. The active footprint is small. The total footprint is enormous.

This creates an asymmetry that conventional inference engines completely ignore. When you run a standard LLM through llama.cpp or MLX, you load the entire model into memory. Every parameter, every expert, every layer — all sitting in RAM whether the current token needs them or not. For a dense model, that's fine. Every parameter participates in every forward pass, so you need all of them resident. For an MoE model, you're keeping 22 billion parameters in memory that any given token will never touch. They're loaded, they're sitting there consuming your RAM, and they do absolutely nothing for the token currently being generated.

TurboFieldfare exploits that gap. It keeps only the shared core (1.35 GB) and the KV cache in memory. Everything else — the 12+ GB of expert weights — stays on your SSD. When the model's router selects an expert for a given token, TurboFieldfare streams just that expert from disk, runs it, and discards it. The next token might need a completely different expert. It streams that one too.

The result: 2 GB of RAM. On a machine Apple sells for $999.

How this actually works (without the hand-waving)

The technique is called expert streaming, and while the concept is simple, the execution is where it gets hard.

Every token in an MoE model goes through a router — a small neural network that decides which experts to activate. In Gemma 4 26B-A4B, each layer has multiple expert subnetworks, and the router picks a handful per token. In normal inference, all experts are preloaded, so routing is just a memory lookup. Instant.

When your experts live on an SSD, routing becomes an I/O operation. You need to read specific byte ranges from a 14 GB file, decompress them (the weights are 4-bit quantized), load them into GPU memory, run the matrix multiplication, and move on — all before the next token needs to be processed. And you need to do this fast enough to maintain a usable decode speed. Miss your timing budget and the user sees a stall. Every layer, every token, the same race against the I/O subsystem.

TurboFieldfare handles this with custom Metal kernels and a purpose-built runtime written in Swift. This isn't a wrapper around llama.cpp or MLX. It's a ground-up inference engine designed specifically for the streaming-expert pattern. The README documents 103 measured experiments across kernels, caching strategies, I/O patterns, prefill optimization, and decode tuning. This is someone who went deep — not "I tried it and it worked" but "I measured every bottleneck and optimized the critical path."

The model weights are quantized to 4-bit (group 64) for experts and shared layers, with an 8-bit router — the router needs more precision because routing decisions are sensitive to small numerical errors. Get the router wrong and the model picks the wrong experts, and output quality collapses. The KV cache runs in FP16. Apple's unified memory architecture is what makes this viable at all — the SSD and GPU share the same memory controller, so the streaming path is shorter than it would be on a system with a discrete GPU separated by a PCIe bus.

There's also a clever caching layer. TurboFieldfare keeps a small pool of recently-used experts in memory, so if the model routes through the same expert multiple times in a sequence (which happens more than you'd think), it doesn't need to re-read from disk. The cache size is configurable, and on machines with more headroom, you can increase the number of expert slots to reduce I/O. On the 8 GB Air, the cache is minimal. On a 24 GB M5 Pro, you can afford to keep more experts hot.

The numbers

On an 8 GB M2 MacBook Air — the base model, the one Apple basically gives away to students — TurboFieldfare hits 5.1 to 6.3 tokens per second. That's not fast by cloud API standards. It's not even fast by local-model standards; a 7B dense model on the same machine would be faster. But it's usable. You can have a conversation with a 26-billion-parameter model on a laptop that cost less than a mid-range phone. The model can reason, follow instructions, and handle multi-turn conversations. It's just not going to win any speed races.

On a 24 GB M5 Pro, the same setup hits 31-35 tokens per second. That's faster than most people type. At that speed, the model feels instant — you hit generate and text appears at reading speed. This is on a machine that, by conventional inference logic, still shouldn't be able to run a 26B model in 2 GB of RAM.

The 14.3 GB model installation is a one-time cost. The installer is itself a piece of engineering — it never materializes the full source checkpoint on disk. Instead, it streams byte ranges from a pinned Hugging Face revision and repacks them directly into TurboFieldfare's custom .gturbo format as they arrive. Peak disk usage during install stays bounded. After installation, you're running entirely offline. No API key, no per-token billing, no data leaving your machine.

The project includes a native Mac app with a real interface, a CLI for scripting, and an experimental OpenAI-compatible loopback server — so you can point existing tools at it just like you would with any local model server. It's text-only; no images, audio, or video. The loopback server accepts function-tool declarations and returns tool calls for the client to execute, which means you could theoretically wire it into an agent pipeline.

Why this matters beyond one cool repo

TurboFieldfare is a single-developer project running one specific model. It's not going to replace vLLM or llama.cpp as a general-purpose inference engine. But it proves something that the local-AI community has been arguing about for a while: the memory requirements for MoE models are not what we think they are.

Most of the "you need 32 GB of RAM to run this model" guidance comes from a world where inference engines load everything upfront. That made sense for dense models. It makes less sense for MoE architectures where 85% of the parameters are asleep at any given moment.

If expert streaming becomes a standard technique — and TurboFieldfare's 103-experiment methodology suggests it's reproducible — then the hardware requirements for running frontier-scale open models drop dramatically. A model like Kimi K3 (2.8 trillion parameters, but only 21B active per token) becomes theoretically streamable. Not fast. Not practical yet — the expert count and model size would make the I/O budget brutal. But theoretically possible on hardware that costs less than a used car.

This is the direction local AI is heading. Not brute-force loading entire models into memory, but intelligent resource management — loading what you need, when you need it, from fast local storage. Apple's unified memory architecture happens to be unusually well-suited for this pattern because the SSD-to-GPU path doesn't cross a PCIe bus. The M-series chips were designed for exactly this kind of tight memory integration, even if Apple didn't have MoE streaming in mind when they built it.

The catch

There are real limitations, and they matter.

Expert streaming adds latency to every token — you're doing disk I/O in the hot path. On the 8 GB MacBook Air, 5 tokens per second is the ceiling, and that's with a model that has relatively few active parameters per token. Denser MoE models would be slower. Non-MoE models get zero benefit from this approach; if every parameter is active on every token, there's nothing to skip.

The SSD wear question comes up a lot. Constantly reading expert weights from flash storage means your SSD is doing a lot more reads than it normally would. Modern NVMe drives are rated for hundreds of terabytes of writes, and reads cause much less wear than writes — but it's worth thinking about if you're running this eight hours a day. The SSD is now part of your inference pipeline, and its endurance is a factor.

And TurboFieldfare is model-specific. It's built for Gemma 4 26B-A4B, not for arbitrary models. Adapting it to another MoE architecture would require significant rework of the Metal kernels and weight packing. The .gturbo format is purpose-built for this one model's architecture. Making it general-purpose would mean supporting different expert layouts, routing mechanisms, and quantization schemes — essentially building a new inference framework.

The bigger picture

What makes TurboFieldfare interesting isn't the speed or the specific model. It's the proof of concept: that with enough engineering, the gap between "what your hardware can theoretically support" and "what you can actually run" is much wider than the default tools suggest.

We've been conditioned to think about local AI in terms of VRAM ceilings. If the model doesn't fit in memory, you can't run it. Full stop. The model card says 16 GB minimum. Your machine has 8. You buy a new machine or you don't run the model.

TurboFieldfare says: what if the model doesn't need to fit? What if it's a streaming resource, not a resident one? What if the 2 GB you keep in memory is just the working set, and the rest lives on your SSD like any other file?

That's a paradigm shift for local inference. And it's coming from a solo developer with a Swift project and 103 measured experiments, not from a well-funded lab with a thousand GPUs and a marketing team. That's the open-source AI story in miniature: someone sees an assumption that everyone accepts, questions it, and builds the thing that proves it wrong.

The implication for anyone building local AI tools is clear. The bottleneck isn't the model size. It's the inference engine's assumptions about how memory works. Change those assumptions, and suddenly the hardware you already own can do a lot more than you thought.

---

If you're into running AI models on your own hardware, CopperRiver is a desktop AI assistant for Mac that connects to open-source models — locally or in the cloud. It browses the web, runs terminal commands, reads files, and automates tasks. Plans start at $9/mo.