by Chris DePuy / September 13, 2026
On September 9, NVIDIA published an official NVFP4 quantization of Z.ai’s GLM-5.3-Flash. Its Model Optimizer recipe leaves the key-value cache at FP8 and compresses only the linear layers inside the transformer blocks — the sparse-MoE shared experts and the dense MLP — from 16 bits down to 4, a step the card sizes at roughly 3.33 times on disk and in memory. The release is aimed at four-GPU Blackwell boards: the card’s own serve commands are tensor-parallel-four, and the stored payload is 204.5 GB, well past the 128 GB of unified memory in a single GB10. Which is why I think the more interesting story on this model is not the vendor checkpoint but what the desk-side community has finished underneath it. Three separate community quantizations now bring the entire 320-billion-parameter model, 18 billion of it active per token, onto one DGX Spark, each with published fidelity receipts attached, and the spread between them is a useful map of what bytes-per-weight actually costs.
The vendor checkpoint, sized honestly
The NVIDIA model card carries a full accuracy panel against the BF16 baseline: GPQA Diamond 0.9211 versus 0.9217, Terminal-Bench 2.1 at 0.8315 versus 0.8258, AA-LCR at 0.7106 versus 0.71, with the softer moves on MMMU-Pro (0.7630 versus 0.7688) and IFBench (0.6054 versus 0.613). Calibration used CNN-DailyMail and NVIDIA’s own Nemotron post-training corpus, and the weights carry an MIT license. The checkpoint’s stored payload is 204.5 GB — a little over 190 GiB — and a community post that same day put the delta over the popular EXL3 4-bit build at about 28 GB larger, which the two cards’ byte counts confirm (175.7 GB for the EXL3/TR3 4bpw checkpoint). On a machine with 128 GB of unified memory, that arithmetic is the whole story: the vendor’s FP4 pack does not fit one desk box at all, and every single-Spark path runs through a community loader that quantizes selectively — routed experts compressed, attention, routers, embeddings and the language head held at source precision.
The tensor-parallel-one build with published receipts
The most complete single-Spark record is a deployment recipe from gitcommit90, which adapts Mia AI Lab’s two-Spark vLLM/EXL3/DFlash foundation to tensor-parallel-one against Turboderp’s 2.05-bits-per-weight checkpoint — 85.23 GB on disk, 79.4 GiB resident. With Inco AI’s DFlash2 drafter at depth seven, structured output decodes at a 64.1 token-per-second five-run median; the shipped default of depth five trades a little structured speed for prose and code, which measured 29.9 and 40.1 tokens per second. The recipe is careful about concurrency accounting — a four-stream run reports 181.9 tokens per second as a sum of active streams and 91.5 as strict wall-clock — and it publishes cold prefill of 786 to 846 tokens per second from 8K through 100K context, a 262,144-token window, and a cold start cut from about 14 minutes to 3 minutes 21 seconds. An independent teacher-logit measurement of that exact checkpoint revision, 51,175 scored positions across the full vocabulary, puts mean divergence at 0.1216 nats with 88.92 percent top-1 agreement against BF16. A separate community run of the same stack measured 25.3 tokens per second at 512-in/400-out, 16 of 16 tool calls passing against the teacher on names and arguments, and 27 of 33 next-token top-1 matches.
The second two-bit route, and an honest shortfall
The other fresh path is a selective TR3 2.0-bits-per-weight conversion published by 0xSero, a 111.35 GB weight payload in 133 shards in which the routed experts from language layers three through 44 carry the compression while attention, routers, the shared experts, embeddings, the language head and the vision tower keep source precision. The author’s launch post reported 18 to 25 tokens per second with vision enabled; the card’s own later measurement series is more conservative, putting a complete 18-request single-stream sweep at 14.00 to 15.48 tokens per second across one-to-200K input lengths and stating plainly that the build never reached its sustained 25-to-50 target. The card is also unusually candid about scope — an abliteration experiment recorded as an unpromoted negative result, a drafter that must be mounted separately under its own non-commercial license, and an exact-match retrieval test that passed at a 200,013-token prompt. That is the documentation standard I would like to see everywhere.
Where the quality ladder lands
Set side by side, the routes form a clean ladder on the two-bit rung and above: the EXL3 K2 pack that established the single-box floor in late August at 0.3346 nats and 0.788 top-1, the new 2.05-bpw build at 0.1216 and 0.8892, the 4-bit checkpoint at 0.0246 nats against the official FP8’s 0.0246, and NVIDIA’s FP4 panel holding within rounding of BF16 on the hardest benches. What I take from the ladder is that the cliff on this architecture sits below about two bits of expert weight, not at two: tool-use fidelity and retrieval survive the compression while prose decode does not — the published single-box prose figures top out near 30 tokens per second, where the drafter is only keeping roughly two to four proposed tokens per verification step, and one recipe’s own notice warns that its speculative drafter is licensed for non-commercial research unless its publisher grants otherwise. I believe the single-GB10 build is now a credible shared-team endpoint for structured and coding agents, where acceptance rates run near one, and still a compromise for prose-heavy interactive work — and at $4,699 for the second box, that compromise is easy to price.
| Route | Checkpoint | Weight payload | Single-stream decode | Quality anchor | Source |
| Community TP1, depth 7 | Turboderp EXL3 2.05 bpw | 85.23 GB | 64.1 tok/s structured | 0.1216 nats, 88.92% top-1 | GitHub recipe |
| Community TP1, same stack | Turboderp EXL3 2.05 bpw | 85.23 GB | 25.3 tok/s, 16/16 tools | 27/33 next-token top-1 | community benchmark |
| Community TP1, TR3 K2 | 0xSero TR3 2.0 bpw | 111.35 GB | 14.0–15.5 tok/s sweep | retrieval verified at 200K | model card |
| Community TP1, MTP k=2 | vcruz305 EXL3 K2 | 97.73 GB (91.0 GiB) | 15.7–16.5 tok/s at 8K | 0.3346 nats, sixcat 84.17 | model card |
| Vendor FP4, TP4 only | NVIDIA NVFP4 | 204.5 GB | not a single-box config | GPQA 0.9211 vs 0.9217 BF16 | model card |
| Route | Memory fit on one GB10 | Leading workload | Leading metric | Best for |
| Turboderp 2.05 bpw + DFlash2 | 79.4 GiB resident, 262K window | Structured output | 64.1 tok/s, 181.9 aggregate | Coding agents, JSON services |
| 0xSero TR3 2.0 bpw | 103.7 GiB payload | Vision and retrieval | 200K verbatim recall | Multimodal review, patient batch |
| vcruz305 K2 + MTP | 91.0 GiB payload | Everyday mixed use | sixcat 84.17 at 64K | Budget single-box floor |
| NVIDIA NVFP4 | exceeds one box | Rack-class TP4 | parity with BF16 on hard benches | Four-GPU Blackwell deployments |
The practical consequence for buyers is that the checkpoint a vendor signs and the checkpoint that fits a desk are now different artifacts, produced by different communities, with only the community loader work — ExLlamaV3’s trellis-based EXL3 kernels and the selective-quant loaders they require — standing between a frontier-class MoE and a single machine. On this model, the two-bit rung has quietly become the default, and the receipts are published.
Several clusters in the same week of notes remain queued for their own posts: the second round of single-Spark Qwen3.8-27B speedups, where a calibrated NVFP4 drafter and a concurrency-tuned build each claim roughly a quarter more throughput and an uncensored NVFP4 mixed quant took the top rank on a community agentic benchmark; the v1.5.0 release of the two-Spark GLM-5.3-Flash EXL3 kit alongside its eight-change sibling update that cuts maximum prompt wait under contention from roughly 120 seconds to 15; the second instalment of the heterogeneous prefill-and-decode split, which doubled that configuration’s context window to two million tokens and published cold-load times at 100K, 500K and 1.25M; the newest single-Spark Qwen3.8-Flash-Next profiles, including a 66.2 tok/s single-stream build and a 273 tok/s eight-stream concurrency run; and the still-open agent shared-memory and Hermes agent-platform notes.