Red Hat AI did something the community wasn’t sure was possible: they adapted DSpark speculative decoding to a non-DeepSeek model. GLM-5.2-FP8 now runs with a DSpark draft model, and the numbers are real. The full validation and model weights are published on HuggingFace (RedHatAI/GLM-5.2-speculator.dspark).
The Breakthrough
DSpark was effectively tied to DeepSeek models — the DFlash parallel draft backbone, the Markov logit-bias head, the confidence head — it all assumed DeepSeek’s architecture. Red Hat’s team trained a 3B-parameter speculator on GLM-5.2’s own regenerated responses (self-play), proving the framework is fully generalizable. The project released in three checkpoints (epoch-1 through epoch-3), with epoch-3 now shipping as the final production model on HuggingFace.
Performance
On 2× DGX Sparks: 19.8 tok/s single-stream. That’s a usable speed for a 5.2T-parameter MoE on two desk-side boxes (HF model card validation).
On 4× B300 reference (from Red Hat’s validation): the progression across training checkpoints tells the story (source):
| Metric | No speculation | Preview | Epoch-1 | Epoch-3 (final) |
|---|---|---|---|---|
| Mean accepted length | 1.0 | 2.18 | 3.49 | 3.97 |
| Decoding speed | 102 tok/s | 139 tok/s | 219 tok/s | ~240+ tok/s |
| Mean acceptance rate | — | 0.411 | — | 0.584 |
| Full accuracy | — | 0.454 | — | 0.613 |
Per-position acceptance on the final checkpoint holds up well across all 7 draft positions — position-1 at 0.78, position-7 at 0.38 — with the Markov logit-bias head effectively mitigating the suffix decay that plagues simpler speculators (HF model card).
Additional Configurations
- 4× Sparks: 15-16 tok/s @ 128K context (community benchmark)
- 3× Spark (REAP-pruned): Full 256K context, viable for long-document workloads (REAP config)
Quick Takeaways
The draft model proposes up to 7 tokens; the large target verifies them in one pass. The final epoch-3 speculator achieves a mean accepted length of 3.97 — meaning nearly 4 tokens confirmed per forward pass on average, or roughly 4× effective throughput on favorable content.
All weights MIT-licensed on HuggingFace (RedHatAI/GLM-5.2-speculator.dspark). Uses vLLM nightly with the speculator PR merged. Zero changes to the base GLM-5.2 model required — just add the draft model and you roughly double your throughput.
Source: HuggingFace model card RedHatAI/GLM-5.2-speculator.dspark (epoch-3 final), Medium article by AI Engineering (July 3, 2026). Validated on 2× DGX Spark (GB10) and 4× B300 reference.