by Chris DePuy / August 4, 2026
When DeepSeek shipped the official DeepSeek-V4-Flash-0731 weights on July 31, the community benchmarks that followed concentrated on single and dual DGX Spark units. Over the past several days a more interesting pattern has emerged as the same checkpoint moves up the cluster ladder, from a single GB10 to pairs, quads, and an eight-node rack. The scaling behavior of this one 284B MoE is becoming a useful map of what the deskside multi-node inference market can actually do in the second half of 2026.
Single and Dual Units: The Baseline
On one DGX Spark, an NVIDIA forum report describes the 0731 build served through a custom engine at roughly 1,000 tok/s prefill and 59 tok/s aggregate across 12 concurrent multi-agent requests.
Moving to a pair changes throughput meaningfully. A widely-circulated 2-node recipe with DSpark speculative decoding, NVFP4 KV cache, and 1M context reports a mean decode of roughly 55 tok/s and a peak above 78 tok/s under high draft acceptance, up from the lower 30s before a shared-expert patch. A second 2-node configuration tracked by Mia AI’s lab measured about 75 tok/s decode at concurrency 1 and roughly 191 tok/s aggregate at concurrency 6. Across these dual-unit reports the shape is consistent: single-stream decode in the 55-80 tok/s band, and aggregate throughput that climbs with concurrency into the low hundreds.
The 4-Node and 8-Node Clusters
The scaling story accelerates beyond a pair. An NVIDIA forum benchmark on a 4-node DGX Spark cluster with DSpark reports roughly 90 tok/s decode and 2,500 tok/s prefill, and a community leaderboard dedicated to Spark hardware, Spark Arena, tracks reproducible vLLM recipes including a 2-node cluster entry.
At the top of the ladder, one operator loaded the 0731 checkpoint across eight DGX Sparks split into four independent tensor-parallel pairs running simultaneously, then pushed a fresh run to 512 and 1,024 concurrent client workflows with the engine staying capped. The same group publishes an open engineering record — cluster build, NCCL and networking tuning, and thermal management — that is notable less for any single number than for demonstrating that an eight-GB10 local AI rack is a real, documented build rather than a one-off demo.
I think the operative metric at this scale is no longer peak single-stream speed but sustained concurrent throughput. The 0731 retraining was aimed at agentic and multi-agent workloads, and the benchmarks that matter for those are aggregate tokens across many in-flight requests — which these cluster runs report in the hundreds to over a thousand for prefill, at the 1M context the checkpoint is designed around.
The Multimodal Contrast: Inkling-Small on Two Units
DeepSeek V4 Flash is a text-only model. For the same two-unit footprint, Thinking Machines’ Inkling-Small — a 276B multimodal MoE with 12B active — has also moved to a dual-Spark configuration. An SGLang run with DSpark speculative drafts measured roughly 34 tok/s single-stream and about 79 tok/s across six concurrent sessions at 1M context, while adding native image, audio, and video input that the DeepSeek model does not carry. The tradeoff is visible in the numbers: the text-only model delivers roughly twice the concurrent aggregate on the same hardware, while the Inkling build buys modality at the cost of throughput. SGLang added official day-0 support for the Inkling family with 1M context, which is part of why the 2-unit multimodal path is now practical at all.
What ties these together is that the open-weight speculative-decoding ecosystem — DSpark for both models, plus the shared-expert and NVFP4-KV patches — is what unlocks the multi-node scaling. As firms such as 650 Group have been tracking in their AI infrastructure research, the center of gravity for agentic inference keeps moving onto small, multi-node local hardware, and the 0731 checkpoint is the clearest example yet of capability scaling with node count rather than rack size.