by Chris DePuy, 650 Group / November 3, 2026.
On November 2, the TensorFold Mac application and the gr8 cluster stack, two community inference projects this blog has followed all autumn, published release notes that move this hardware tier toward installed software. Ash Hart, the maintainer of the TensorFold Mac application, announced version 1.0.0, “our first stable Mac app release, with Linux and Windows to follow shortly”. Meanwhile, the gr8 team announced multi-model support on four-DGX-Spark clusters. Read side by side, the two notes describe one transition: inference engines that spent the summer as command-line projects are becoming packaged products with installers, update channels and more than one model behind one address.
What did the TensorFold and gr8 release notes ship this week?
The two releases answer different needs. TensorFold 1.0.0 serves the single operator who wants one capable assistant on a laptop, engine updates included (project page). gr8’s multi-model cluster serves the office where several users with different jobs share four boxes through one address. As of November 2026 both are the projects’ own releases. Even so, every figure below is a maintainer’s own measurement, taken on the maintainers’ machines, pending independent replication.
TensorFold: The Application and the Engine Are Two Update Channels
The TensorFold 1.0.0 announcement lists five changes, and the two structural ones sit underneath the visible interface. The application ships with engine 1.2.0. Users who want the newer engine can opt in to 1.2.1 under Models, Details, Engine updates. The announcement adds that “Activation waits until local work has finished,” so an engine update does not interrupt a running generation. That separation of application from engine is the release-engineering decision the whole announcement turns on. In other words, the interface can move at its own pace while the engine keeps a supported path. The previously bundled engine stays as a fallback.
The visible changes push the TensorFold assistant into everyday use. A compact ask bar picks the window to share, previews what leaves the machine and pulls text locally where available. Closing the main window keeps the application serving requests from behind a menu-bar icon. The notes also cover connection health and the inbox, so account status and provider errors surface in one place. Finally, Portuguese joins the supported interface languages. The project page hosts the current release notes.
gr8: Several Models, One Fabric
The multi-model announcement states that four DGX Sparks now carry three models to choose from, and adds: “Each model is accelerated with EXLR8 – faster decode and lightning fast boot (< 30s!)."
The engine behind the claim is gr8. Its README calls it “a from-scratch CUDA inference engine for the full GLM 5.3 model.” The README describes a Mixture of Experts that distributes 256 experts, built on MLA attention and a single MTP layer. The model also spans three to six DGX Spark (GB10) boxes with tensor parallelism across the whole group. Indeed, the README is explicit about the runtime it replaces: “It uses no vLLM, PyTorch or NCCL at serve time.” Each box runs one engine process, and “the ranks talk over RoCE v2” over a dedicated verbs fabric. An OpenAI-compatible API on the head box is what lets several users share the cluster through one address, each with different jobs.
The requirements section of the same README reads like a switch bill. It calls for three to six GB10 boxes with 128 GB of unified memory each and one rank per box. Each box is wired for RoCE v2 over two network interfaces. All of them, in turn, connect through a single 200G switch. For smaller offices the README supplies an alternative: at TP=3 or TP=4, the ring shape runs with no switch at all. Then each box’s first port cables to the next box’s second. The README puts the cost at roughly 10% of one stream’s decode. It is 4-5% once four streams run. Every box runs the same command. Each box fetches its part of the weights once. It then boots from its local cache in about 40 seconds.
The Two Projects’ Own Measurement Tables
The gr8 side supplies the tables here; the TensorFold 1.0.0 notes ship packaging changes rather than benchmark numbers. The engine’s own results table comes from 2026-10-24 measurements with a public harness at the mean of two fresh boots. It lists single-stream decode of 41.1 tokens per second on code and 31.7 on prose at TP=3. Decode rises to 52.1 and 34.2 at TP=4. Cold prefill of an 8K prompt runs 635 tokens per second at TP=3 and 1,357 at TP=4 (gr8 README). The key-value pool holds 485,376 tokens at TP=4.
The sixth-processor caveat, though, belongs to the TP=6 column. The README footnote records that “one of the six boxes was capped at 2200 MHz”. It also records that the machine “powers off under sustained load at its full clock”. That clock-cap story is the subject of this blog’s November 1 report on the Spark thermal playbook. The capped column therefore stays out here.
The Flash model on Sparks has history here: in September, this blog printed the table TensorFold published for GLM-5.3-Flash on two Sparks, measured against vLLM. The faster-boot claim refers to the companion recipe for the Flash variant, glm-5.3-flash-exlr8-gx10. Its README also prints one table of its own measurements, taken 2026-10-19. Clocks sit locked to 2200 MHz per tensor-parallel width, per the recipe’s own harness:
| Width | decode code | decode prose | prefill 8K | prefill 32K |
| 1 box (k2) | 53.0 | 29.5 | 495 | 523 |
| 2 boxes (k3) | 84.2 | 50.1 | 2,609 | 2,687 |
| 3 boxes (k4) | 105.8 | 57.8 | 3,002 | 3,391 |
| 4 boxes (k4) | 128.0 | 72.5 | 4,393 | 4,190 |
All values in tokens per second for the Flash side of the week TensorFold and gr8 split between them. Source: recipe README, captured at commit e302622.
The Quantization Ladder Behind the EXLR8 Formats
The quantization ladder in that table has three rungs the recipe documents, one for each width: k2 at roughly 2 bits per weight, k3 at 3 and k4 at 4, all held in one assembled release and selected at runtime, because only the configuration padding is topology-specific. Moreover, the rationale the recipe gives for the EXLR8 formats: decode gains of +3 to +8% (code +7.7%, prose +3.0%), negative log-likelihood against bfloat16 equal or better (+0.0034, +0.0017 and +0.0020 on the three test sets), and about 15% more key-value pool room.
The ladder is the Spark recipe’s own arithmetic; the TensorFold half of the week ships none of it, since its changes are packaging and engine channels, not quantization tables. The open item on the list is prefill, which is currently 7.7% behind at 32K and 128K prompts, “the one open gap, entirely in the routed MoE kernel,” with the note “Work in progress” appended, per the recipe README.
Choosing a Quantization: bit325 and the Fit Limits
Asked which quantization to install, the gr8 maintainer answered in a reply in the same thread: “I recommend our bit325 quant – it’s the best tradeoff that allows it to run, and our quality is comparable to ~3.5ish bits of EXL3. To get our 4-bit quant (~4.25-bit+ equivalent) you def need 5-6.” The README’s variants table then files the details. Home at 3.25 bits per weight is the default build at TP=4 and TP=6. Bit325 costs 1.5 to 7% lower KL divergence than home for about 2.5% slower decode. The table lists the 4-bit up1 build at about 4.0 bits per weight. The five- or six-box requirement for that build is the maintainer’s own, from the reply quoted above.
However, one build constraint remains from the same table: at TP=3, home and bit325 would not fit — each rank would hold 108.7 GB of weights. Still, the maintainer’s own words for the new stack are an expectation, not a result: “We expect that the prefill and decode on GLM 5.3 flash is going to eventually hit the same level as the vLLM-based recipe (code ~128 tok/s, prefill ~4k tok/s)!” (the follow-up post). Thus, that target is the recipe table’s own published 128.0 in the four-box row (TP=4), not a measured result of the new engine.
What the TensorFold and gr8 Release Notes Add Up To
Ultimately, the point of the week is not any single number in the tables above. Rather, it is that both projects shipped the parts that make a project usable by people rather than by its author: an application bundle with a stable version, an update channel with a rollback path, a documented quantization ladder with fit notes for each tensor-parallel width, and more than one model served behind one API. gr8 already served the full 753-billion-parameter model on four Sparks, as this blog reported on October 28; the November 2 note adds the multi-tenant feature, which is the kind of change that turns a demonstration bench into a production line.
Meanwhile, TensorFold 1.0.0 does the same on the other end of the rack, where one box is a laptop. The remaining rough spots are the ones the release notes disclose openly: a prefill regression the maintainers call work in progress, and platform builds the announcement marks as coming soon rather than shipping. Both dates, as of November 2026, are the projects’ own. As firms such as 650 Group have been tracking in their AI infrastructure research, the desk-side inference market keeps moving from single-user to multi-user operation, and the release notes above are the evidence on the record.