Tag: Local-Inference
-
TensorFold 1.0.0 and Multi-Model gr8: Two Release Notes Date the Desk-Side Stack
by Chris DePuy, 650 Group / November 3, 2026. On November 2, the TensorFold Mac application and the gr8 cluster stack, two community inference projects this blog has followed all autumn, published release notes that move this hardware tier toward installed software. Ash Hart, the maintainer of the TensorFold Mac application, announced version 1.0.0, “our…
-
The RTX 30-Series Tensor Unlock: What the Measured Gains Cover and What They Miss
by Chris DePuy, 650 Group / October 29, 2026 An open-source Linux kernel module restores the full FP16/BF16 tensor rate on NVIDIA’s consumer RTX 30-series cards. The first complete benchmark suite in the lane, published this week by an independent operator, measured the matrix-side gain at 50 to 60 percent, and every report in the…
-
New GLM-5.3 on Four DGX Sparks: Results and Media Studio
On October 16, a community recipe put the full GLM-5.3 on four DGX Sparks, at tensor parallelism 4. The model is GLM-5.3 (753B), and the four NVIDIA DGX Spark units run it. Underneath is gr8, a from-scratch inference stack for the GB10 processor. The same weekend, keys/Wasserman AI Studio announced release 2.8.0 for Mac and…
-
DeepSeek Open-Sources DeepInfer, NVIDIA Answers Quietly With Flare
by Chris DePuy, 650 Group / October 22, 2026 In the early hours of October 20, Pacific time, DeepSeek announced DeepInfer, its in-house inference framework, as fully open source under the Apache 2.0 license. DeepInfer is the stack that powers DeepSeek-V4.1-Flash at scale in the company’s own production services. The four-post thread laid out the…
-
The Spark Fleet Learns to Run Itself
by Chris DePuy, 650 Group / October 21, 2026 The last month of desk-side serving stories were about engines. Athena put two frontier models on one 128 GB box at 262K context with a 141,519-token conversation restored from disk in 2.1 seconds; Imprint turned that same restore trick into a standalone context-cache service with a…
-
New TensorFold LLM Model Serving Software Dramatically Improves Decode Speed on Macs and Sparks
by Chris DePuy, 650 Group / September 27, 2026 Two repositories shipped from one maintainer this week and made the same promise in two different places. On September 25 and 26, a developer publishing under the name Ash Hart released TensorFold, a local inference engine for Apple Silicon and NVIDIA GPUs, and Imprint, a tool…
-
New NVFP4 Checkpoints on Four DGX Sparks: The Results
Between September 9 and September 16, NVIDIA published first-party NVFP4 checkpoints for two frontier open-weight models, both MIT-licensed. One is a build of Z.ai’s GLM-5.3-Flash; the other is a build of DeepSeek’s DeepSeek-V4.1-Flash. Both labs produced them with NVIDIA Model Optimizer and validated both on rack-mount Blackwell silicon rather than on the desk. That distinction…
-
DeepSeek-V4.1-Flash on Four DGX Sparks: The Profile Day One Only Configured Now Boots, Serves and Passes a Million-Token Needle Test
by Chris DePuy / September 14, 2026 Late Sunday evening Pacific time, Mia AI Lab pushed a four-rank serving profile for DeepSeek-V4.1-Flash to its DGX Spark recipe, the same lab whose September 13 post on X put the model on four DGX Sparks and invited readers to run the stack themselves. The significance is not…