inference

10 posts tagged “inference”.

1. Our Chunked-Prefill Fix Was 72x Slower. Every Test Passed. (2026-09-07)
2. We Wrote Our Own Inference Engine. It Started at 0.9 tok/s. (2026-09-06)
3. We Evaluated a Faster Inference Engine and Rejected It (2026-09-03)
4. 2x Decode From Block-Diffusion Drafting (2026-09-03)
5. Our Prefix Cache Died at Token 1,067 (2026-09-02)
6. Our Baseline Was the Three Worst Samples in the Log (2026-09-02)
7. The Health Checker Was Manufacturing the Stalls It Detects (2026-09-01)
8. One Slow Request Locks Out Everything Else (2026-08-17)
9. What Thinking Mode Actually Costs (2026-08-11)
10. We Spent a Week Tuning Prefill. The Bottleneck Was Decode. (2026-08-11)