Moonshot AI released Kimi K3 on July 16 — the largest open-weight model ever published at 2.8 trillion total parameters (Moonshot AI announcement). This isn’t a deskside model. It’s the new ceiling the quantization community needs to tackle.
Architecture at a Glance
| Spec | Value |
|---|---|
| Total parameters | 2.8 trillion |
| Active per token | ~50B (16 of 896 experts) |
| Context window | 1,048,576 tokens |
| Modalities | Text + vision |
| Variants | K3 Max, K3 Swarm Max |
| Open weights | July 27 (promised) |
Architecture details from Moonshot AI’s technical blog.
Key Innovation: Delta Attention
Kimi Delta Attention (KDA) delivers 6.3× faster decoding at 1M context vs. standard attention by modifying how the model computes long-range token relationships (Moonshot AI technical blog). Paired with Attention Residuals (skip connections within attention layers), it addresses the fundamental cost of MoE inference at extreme context lengths.
Benchmarks (Self-Reported, Directional)
All benchmarks below are self-reported by Moonshot AI (source). Independent validation will follow when weights arrive July 27.
| Task | Score |
|---|---|
| GPQA Diamond | 93.5% |
| BrowseComp | 91.2% (best published at release) |
| Terminal-Bench 2.1 | 88.3% |
| MCP Atlas | 84.2% |
| LMSYS Frontend Code Arena | #1 (passed Claude Fable 5, up from #18 for K2.6) |
What It Means
Moonshot claims K3 beats Claude Opus 4.8 max and GPT-5.5 high on most evaluated tasks and competes with Claude Fable 5 and GPT-5.6 Sol on their strongest benchmarks (Moonshot AI blog). Independent validation needed — weights arrive July 27.
The practical take: This is a 896-expert MoE with ~50B active parameters. For local inference, it’s cluster territory. But it sets the target for what quantization needs to handle — and it’s proof that the open-weight frontier has moved past what anyone predicted at the start of 2026.
Sources: Moonshot AI technical blog (Kimi K3 announcement), ChatForest Builder’s Log, Tom’s Hardware, Graphify Guides. Weights arrive July 27 via HuggingFace.