Tag: Local-Inference
-
New TensorFold LLM Model Serving Software Dramatically Improves Decode Speed on Macs and Sparks
by Chris DePuy, 650 Group / September 27, 2026 Two repositories shipped from one maintainer this week and made the same promise in two different places. On September 25 and 26, a developer publishing under the name Ash Hart released TensorFold, a local inference engine for Apple Silicon and NVIDIA GPUs, and Imprint, a tool…
-
NVIDIA’s Official NVFP4 Checkpoints for DeepSeek-V4.1-Flash and GLM-5.3-Flash: Four DGX Sparks Read the Receipts
by Chris DePuy / September 22, 2026 NVIDIA: what the conversion actually changes The DeepSeek checkpoint converts only the ordinary routed experts — projections w1, w2 and w3 across 384 experts in 40 layers — from the source MXFP4 format to NVFP4 weights and activations at group size 16, per the model card; attention, shared…
-
DeepSeek-V4.1-Flash on Four DGX Sparks: The Profile Day One Only Configured Now Boots, Serves and Passes a Million-Token Needle Test
by Chris DePuy / September 14, 2026 Late Sunday evening Pacific time, Mia AI Lab pushed a four-rank serving profile for DeepSeek-V4.1-Flash to its DGX Spark recipe, the same lab whose September 13 post on X put the model on four DGX Sparks and invited readers to run the stack themselves. The significance is not…
-
Day One on Three and Four Sparks: Community Recipes Put DeepSeek-V4.1-Flash on the Desk
by Chris DePuy / September 12, 2026 On September 10, DeepSeek released DeepSeek-V4.1-Flash, a multimodal mixture-of-experts model with a 552-billion-parameter backbone and a context window of one million tokens. The launch kits for NVIDIA’s desk-side DGX Spark machines arrived the same day: a four-machine stack on vLLM was serving requests by 9:17 Eastern time, roughly…
-
Nemotron 3.5 Lightning: NVIDIA Carves Out an Agent Execution Layer for the Desk-Side
by Chris DePuy / August 12, 2026 NVIDIA this week released Nemotron 3.5 Lightning, an open 30B MoE model with just 3B active parameters, and I think it is the clearest signal yet that the deskside inference market is splitting into two distinct tiers. The model is built for the high-volume execution layer of long-running…
-
DFlash2 Beats DSpark for Qwen3.8-27B on One DGX Spark — and Jumps to Apple Silicon
by Chris DePuy / August 20, 2026 A week ago the Qwen3.8-27B decode race on a single NVIDIA DGX Spark was a contest between DSpark and MTP. This week the leading recipes on the same hardware have switched to a different engine: DFlash2, the block-diffusion speculative decoder that first shipped with DeepSeek, is now beating…
-
GLM-5.3-Flash Day Two: DFlash 2 Attacks the Decode Path While 1M-Context KV Moves Onto Two DGX Sparks
by Chris DePuy / August 28, 2026 A day after Z.ai opened weights on GLM-5.3-Flash, the desk-side community pushed the model past what I thought was its practical limit on small hardware. Two bottlenecks — the decode path and the KV cache — were attacked in parallel, and both moved in the same 24 hours.…
-
DeepSeek V4 Flash 0731 on the Multi-Node DGX Spark: From One Pair to an Eight-Node Cluster
by Chris DePuy / August 4, 2026 When DeepSeek shipped the official DeepSeek-V4-Flash-0731 weights on July 31, the community benchmarks that followed concentrated on single and dual DGX Spark units. Over the past several days a more interesting pattern has emerged as the same checkpoint moves up the cluster ladder, from a single GB10 to…
-
Poolside’s Laguna S 2.1 118B Punches Above Its Weight
by Chris DePuy / July 28, 2026 This week, the local AI community has been running Poolside’s latest open-weight model, Laguna S 2.1, on everything from single DGX Sparks to 3090 Quad workstations. Released on July 21, Laguna S 2.1 is a 118B total parameter Mixture-of-Experts model with 8B activated parameters per token, a 1M-token…