by Chris DePuy / August 28, 2026
A day after Z.ai opened weights on GLM-5.3-Flash, the desk-side community pushed the model past what I thought was its practical limit on small hardware. Two bottlenecks — the decode path and the KV cache — were attacked in parallel, and both moved in the same 24 hours. One team served the model’s full 1,048,576-token context across just two DGX Sparks with a memory-quantized KV cache, and a light-weight draft model called DFlash 2 delivered the first working speculation deployment of GLM-5.3-Flash on GB10, roughly doubling single-stream decode. The story is less about any single number and more about where the ceiling actually sits: not in weights, but in the memory and kernel path that service long context.
Two Sparks, the Full Million Tokens
The model’s native context is a million tokens, and the first recipe to serve the whole thing on paired GB10 nodes does it with NVFP4 weights alongside an NVFP4-packed MLA KV cache. The cache density matters more than the weights here: the packed record comes in at 368 bytes per token per layer, versus 656 for fp8 and 1,152 for bf16, which is what lets a full-million-token request hold on 121.7 GiB unified memory per node. The operator built a KV pool of roughly 1.22 million tokens, or 1.17x a single full-million request, and verified passkey retrieval at 1,043,880 prompt tokens. Decode runs around 22-27 tok/s single-stream, with C8 aggregate hitting 84 tok/s on code and 75 tok/s on prose.
What impressed me was prefill scaling across the context ladder. Prefill throughput held roughly flat — 1,050 to 1,290 tok/s from 33K out to 1.04M tokens — which suggests the model’s sparse attention path scales as designed rather than degrading into quadratic work. The cost shows up in time to first token: 889 seconds at 1.04M tokens. So the million-token ceiling on two Sparks is real, but at roughly 15 minutes of prefill before the first token, it is a throughput ceiling for batched or cached work, not a latency story.
The same 24 hours pushed the KV pool on four Sparks even further. One operator grew the fp8 KV pool to 5,033,164 tokens, enough for 4.8 concurrent requests each at the full million-token context — the 320B/18B model in NVFP4 on what he priced at about $16K of desk hardware. Separately, another submitted the first documented recipe running the official FP8 checkpoint (not the community NVFP4 quant) at 262K context on four Sparks.
DFlash 2 Fixes What MTP Could Not
The more consequential development is on the decode side. GLM-5.3-Flash ships an in-checkpoint MTP head, but the 1M-context recipe measured MTP as decode-neutral: acceptance was healthy at 73%, yet throughput barely moved because the multi-token verify step on the sparse-MLA path costs about as much as the accepted tokens save on GB10. The intended fix was an external draft model rather than deeper MTP.
That fix landed within a day. Inco AI published DFlash 2, a 1B-parameter block-diffusion drafter for GLM-5.3-Flash that predicts a whole block of tokens in a single pass, and it is now served on two DGX Sparks: 46.9 tok/s single-stream versus 21.8 with MTP-4, a 2.15x improvement at 74.1% draft acceptance, at zero KV cost because the drafter’s layers share the MLA tensors on GLM’s own fast path. On the structured and tool-argument output that agents actually generate, the same operator measured roughly 54-61 tok/s, about 2.5-2.8x the MTP-4 baseline. Inco’s own numbers on four GB300 nodes put DFlash 2 at up to 2.79x over autoregressive at concurrency 1 (MATH-500: 438.9 tok/s), and ahead of GLM’s native MTP on every task tested. The drafter carries a research-only CC BY-NC-ND 4.0 license.
There is a caveat worth stating plainly. Getting the drafter onto GB10 took nine boots and four patches, and the setup notes warn that the generic uniform-page KV path cannot serve GLM-5.3 with a drafter attached — the drafter has to share the model’s custom fast path to run at all. This is not a drop-in DFlash 2 deployment; it is a port that only reproduces because the operator shipped the full patch stack with it.
The KV Cache Is the Real Constraint
Taken together, the day-two results point to where the bottleneck actually lives. GLM-5.3-Flash’s active-parameter footprint is small enough for desk hardware — 18B active of 320B total — but its KV footprint scales with context, and at a million tokens even fp8 KV strains a pair of Sparks. The winning configuration pairs NVFP4 weights with NVFP4 KV, and the DFlash 2 result shows decode can be doubled without spending any additional KV budget because the drafter loans the target’s cache. As firms such as 650 Group have been tracking in their AI infrastructure research, the desk-side inference tier is becoming a memory-and-KV engineering problem as much as a compute one — and the community is moving on both fronts within hours of each release.