by Chris DePuy, 650 Group / October 22, 2026
In the early hours of October 20, Pacific time, DeepSeek announced DeepInfer, its in-house inference framework, as fully open source under the Apache 2.0 license. DeepInfer is the stack that powers DeepSeek-V4.1-Flash at scale in the company’s own production services. The four-post thread laid out the scope in short order: high-performance kernels and an optimized engine, distributed inference, and speculative decoding, followed by a separate release of deepseek-recipe, the production token encoding and decoding component DeepSeek describes as aligned with the processing logic behind the official DeepSeek API. Thirteen hours later, a community tracking account surfaced Flare, a local inference server NVIDIA built for its own compact workstation silicon, including the DGX Spark. Two serving stacks, published by the company that owns the model and the company that owns the silicon, inside a single day. I think the day marks the point where local serving stopped being a community-patch exercise and became a first-party product category.
What did DeepSeek open source on October 20?
DeepSeek released DeepInfer, the inference framework behind DeepSeek-V4.1-Flash’s production deployment, together with deepseek-recipe, its production tokenization component. Which stack fits which buyer depends on hardware and model roster. Operators running DeepSeek models on NVIDIA SM100-class data center GPUs or Huawei Ascend 950 NPUs get a purpose-built engine in DeepInfer. Buyers with a desk-side box — an RTX PRO 6000 Blackwell workstation or a DGX Spark on SM120/SM121 silicon — are the audience for NVIDIA’s Flare instead. Teams needing broad model and hardware coverage still have vLLM, and both new stacks position themselves against that baseline rather than replacing it.
What is inside DeepInfer, and what is missing?
The DeepInfer repository reads as a production framework distilled to its serving core. Supported models currently list exactly one: DeepSeek-V4.1-Flash, with the DeepSeek-V4.1-Flash-1017 checkpoint pulled from Hugging Face by default. The feature set covers optimized kernels — FlashMLA, DeepGEMM, DeepEP, TileKernels, DeepSelect, and custom TileLang kernels — plus attention data parallelism with balanced context parallelism, MoE expert parallelism with EPLB balancing, and a DSpark speculative path with confidence-scheduled verification where environment flags control the MTP varlen path and the maximum draft length, documented at a default of five. Decoder SWA bounded replay trims what the decoder must re-process on a long prompt, keeping only a bounded suffix of context — kept long enough to span the SWA window and the furthest backtrack the decoder can take. Asynchronous scheduling, on by default, lets the host form batches while the GPU executes, and the release adds batch-invariant inference, deterministic sampling under a fixed seed, and structured outputs expressed as JSON schema, EBNF, or regular-expression constraints.
The boundaries are stated as plainly as the features. The repository targets NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, requires CUDA 13 or CANN 9.2 respectively, and explicitly does not include a KV cache system, prefill/decode disaggregation, or load balancing across data-parallel replicas — the README notes most of the required engine functionality is already implemented, but a complete inference service needs those pieces from elsewhere. The published performance charts came from a four-GPU SM100 node on the prefill side and a 64-GPU node on the decode side, each within one scale-up domain, and a sweep script ships for reproduction. The acknowledgments are equally direct: the codebase descends primarily from vLLM, with SGLang and TensorRT-LLM also credited as influences. One detail in the release is easy to overlook and analytically loud: DeepSeek shipped the same engine for NVIDIA CUDA and for Huawei’s CANN 9.2 stack, hedging its serving layer across both silicon ecosystems from day one.
What does DeepInfer mean for DGX Spark operators?
Strictly, nothing yet runs: DeepInfer lists SM100 and Ascend 950 hardware, and the DGX Spark’s GB10 is SM121 silicon, so the framework does not cover the desk-side box in its current release — even though the model’s four-Spark serving profile is by now well documented. The Spark connection is a performance argument rather than a port. An independent operator’s reaction post, published about ninety minutes after DeepSeek’s announcement, framed the economics: on four DGX Sparks, community measurements as of October 2026 already show 48.5 tokens per second for one user under stock vLLM climbing to 93.6 under TensorFold, with the post’s own framing being that cheaper serving means more tokens out of the same hardware; the open question is how quickly DeepInfer’s kernel and scheduling ideas port into those Spark stacks and lift throughput per watt and per dollar on an identical rig. The deepseek-recipe release adds a second, quieter hook: because the component aligns token processing with the official API’s processing logic, a hybrid deployment that splits work between a local rig and DeepSeek’s hosted API gets the same token behavior on both sides. That detail lowers the cost of treating local hardware as an overflow tier for a hosted service rather than a separate island.
What is NVIDIA Flare for DGX Spark?
Where DeepInfer reaches up toward the data center, Flare reaches down to the desk. The Flare repository describes a local inference server for SM120/SM121 GPUs — the RTX PRO 6000 Blackwell and the DGX Spark — aimed at extended conversations, coding agents, and heavy repeated tool calls, with a design that splits the stack in two. At build time, a component called Ignition leans on the standard PyTorch stack — Transformers plus FlashInfer/b12x — to load the model, tune its kernels, compile the forward pass, and pre-capture CUDA graphs before packing everything into a runtime bundle. Serving then happens in Helios, a Rust runtime that needs no Python interpreter because compilation and graph capture are already done. Buffers and execution capacity get planned before any request arrives, and the server treats a conversation’s whole state — the KV cache and any recurrent components together — as one unit, so resuming a thread or branching from it keeps that state intact. The split of responsibilities is deliberate: per-model recipes carry the model’s computation, and the shared runtime handles scheduling, cache storage, and requests. The README is unusually direct about the trade-off: vLLM’s reach across many more models, hardware targets, and deployment patterns is far wider, and Flare deliberately narrows to a small set of models and local workloads so that the model-prep layer, the kernels, memory management, and the scheduler are designed as one piece. The repository’s own documentation fills in the serving specifics: KV storage lands in FP8 E4M3 for all recipes — draft-model caches included — while a few models keep higher-precision selector keys, and the prepared recipe roster covers the Qwen3.8 family, GLM, and DeepSeek. Packages for Qwen at TP1 and TP2, and GLM at TP4, are documented as exercised end to end, and the Qwen3.8 Flash recipe exports as many as five MTP draft tokens in one round. NVIDIA is equally explicit about maturity: the README describes Flare as a research prototype made available for technical collaboration and early feedback, and states that it is not a replacement for production inference engines like vLLM.
DeepInfer vs Flare at a glance
| Attribute | DeepInfer (DeepSeek) | Flare (NVIDIA) |
| Builder | DeepSeek-AI | NVIDIA |
| Target hardware | NVIDIA SM100 GPUs; Huawei Ascend 950 NPUs | SM120/SM121: RTX PRO 6000 Blackwell, DGX Spark |
| Serving runtime | Python engine, built primarily on vLLM | Rust runtime (Helios), no Python at serve time |
| License | Apache 2.0 | Apache 2.0 |
| Ships today | Optimized kernels, MoE EP/EPLB, DSpark speculative decoding, SWA bounded replay, async scheduler, structured outputs | CUDA-graph runtime package from Ignition build, explicit memory/state planning, focused model recipes |
| Left to the operator | KV cache system, prefill/decode disaggregation, replica load balancing | Breadth: coverage limited to selected models and prepared capacities |
Read together, the two releases describe the same convergence from opposite ends. DeepSeek is optimizing serving for its own models at data-center scale — and, notably, for a second silicon vendor — while NVIDIA is optimizing serving for its own compact workstation silicon, where the memory and bandwidth budget is the binding constraint. The overlap sits in the middle: long conversations, agentic tool use, and KV-state management are the declared design targets in both READMEs, which is a fair map of where local inference actually gets hard. The practical consequence for operators is a serving layer that is moving from patched open-source engines to stacks with a named owner and a stated contract, and the open question is how quickly the kernel-level ideas in DeepInfer reach the SM121 cards the community actually runs. As firms such as 650 Group have been tracking in their AI infrastructure research, the serving layer for local AI is consolidating around first-party stacks from the model owners and the silicon owners at the same time, and deskside buyers are the beneficiaries of the competition between them.