Mixture of experts (MoE) is the architecture trick that lets a model be enormous on paper and affordable in practice. Instead of one dense block of weights that every token passes through entirely, an MoE model splits much of its capacity into many parallel experts — smaller sub-networks — and a lightweight router picks a few of them for each token. The result is a model with, say, 1 trillion total parameters that only activates 30–40 billion per token: frontier-scale knowledge at mid-size compute cost.
How routing works
At each MoE layer, the router scores every expert for the incoming token and sends it to the top few (often 2–8 of 64–256). Different tokens take different paths — one flows through experts that fired on code-like patterns, the next through ones tuned by math-heavy training. “Experts” is a loose metaphor: they aren’t hand-assigned topics but specializations that emerge during training, and the router learns alongside them. Training has to keep loads balanced (an expert nobody routes to is wasted capacity), which is a genuine engineering headache — but one the big labs have largely solved.
Why it took over
Model quality scales with parameters, but inference cost scales with active parameters — MoE decouples the two. That’s why most frontier and near-frontier LLMs in 2026 are MoE, and why the architecture is the signature of the strongest open-weight models: DeepSeek V4 Pro (~1T total / ~32B active), Kimi K2.7 Code (1T / 32B) and MiniMax M3 (230B / 10B) all ride the same lever, which is how they undercut dense rivals on price while matching much of their quality.
The catch
The compute discount doesn’t extend to memory: all experts must sit in VRAM, because any token might route anywhere. A 1T-parameter MoE runs fast but still needs cluster-class hardware to hold — self-hosters typically pair MoE with quantization to shrink the footprint. For API users the trade is invisible; you just see the result — big-model quality at small-model prices, which is exactly the point.