robotnurse./benchmarks — the citable index
Every benchmark, transcribed verbatim.
One page with every measured figure from RobotNurse's field notes — peak matmul of 22.91 TFLOP/s on a healthy Mac Studio M3 Ultra against 4.83 TFLOP/s on a sick one (4.7× slower), read bandwidth 535 → 597 GB/s, and the rest. Sortable, filterable, and source-cited per row.
✎
Every figure on this page is transcribed verbatim from robotnurse.io's own posts — nothing estimated or inferred. Each row cites its source post; cells are blank where the source did not report that quantity.
22.91TFLOP/s · peak matmul, healthy Mac Studio M3 Ultra4.83TFLOP/s · same box, sick — 4.7× slower535→597GB/s read bandwidth · before → after reboot
[ the full data set — 45 rows ]
click a column header to sort
| Category | Model / measurement | Hardware / quantization | Throughput / value | Memory / VRAM | Context | Settings / notes | Source |
|---|---|---|---|---|---|---|---|
| Throughput | (bare matmul probe) | 8-bit quantized · Mac Studio M3 Ultra, 80 GPU cores, 256 GB ("Studio B" — healthy) | 22.91 TFLOP/s (15.93 ms) | — | — | median of 10, forced eval | 11-benchmark-the-hardware-first |
| Throughput | (bare matmul probe) | 8-bit quantized · Mac Studio M3 Ultra, 80 GPU cores, 256 GB ("Studio A" — sick, 4.7× slower) | 4.83 TFLOP/s (75.61 ms) | — | — | median of 10, forced eval | 11-benchmark-the-hardware-first |
| Context | (same model, same server) | Studio B (M3 Ultra, 80-core GPU, 256 GB) | 373.7 tok/s prefill (TTFT 69.5 s) | — | 25,970-token prompt | — | 11-benchmark-the-hardware-first |
| Context | (same model, same server) | Studio A (M3 Ultra, 80-core GPU, 256 GB) | 118.2 tok/s prefill (TTFT 219.8 s) | — | 25,970-token prompt | — | 11-benchmark-the-hardware-first |
| Throughput | (read bandwidth probe) | Studio A dirty | 94 GB/s (erratic) | — | — | — | 11-benchmark-the-hardware-first |
| Throughput | (read bandwidth probe) | Studio A clean / Studio B | 535 GB/s / 538 GB/s | — | — | — | 11-benchmark-the-hardware-first |
| Throughput | (small matmul 1024³) | Studio A dirty / clean / B | 0.13 / 2.81 / 4.65 TFLOP/s | — | — | — | 11-benchmark-the-hardware-first |
| Throughput | (quantized matmul) | 8-bit · Studio A clean / Studio B | 7.0 (steady) / 22.9 (steady) TFLOP/s | — | — | — | 11-benchmark-the-hardware-first |
| Throughput | (matmul, unquantized) | fp16 · Studio A clean | 7.18 TFLOP/s | — | — | — | 11-benchmark-the-hardware-first |
| Throughput | (post-reboot probes) | fp16 · Studio A before → after (twin) | GPU 7.23 → 22.61 (twin 22.60) TFLOP/s | — | — | warm reboot | 11-benchmark-the-hardware-first |
| Throughput | (post-reboot probes) | fp16 · Studio A before → after (twin) | CPU 3.10 → 5.71 (twin 5.55) TFLOP/s | — | — | warm reboot | 11-benchmark-the-hardware-first |
| Throughput | (post-reboot probes) | Studio A before → after (twin) | read bandwidth 535 → 597 (twin 538) GB/s | — | — | warm reboot | 11-benchmark-the-hardware-first |
| Throughput | (post-reboot probes) | Studio A before → after (twin) | small matmul 1024³ 2.81 → 6.10 (twin 4.65) TFLOP/s | — | — | warm reboot | 11-benchmark-the-hardware-first |
| Context | (end-to-end prefill) | Studio A before → after (twin) | 118.2 → 377.2 tok/s (twin 373.7) | — | 25,974-token prompt | — | 11-benchmark-the-hardware-first |
| Throughput | (server warmup ladder) | Mac Studio (M3 Ultra) | 08-23: 314.6 (512 tok) / 330.0 (2560 tok); 08-24: 201.0 / 374.8; 08-31: 359.3 / 370.1; 09-02 17:21: 122.8 / 107.3; 09-02 20:16: 35.9 / 28.5 tok/s | — | 512-token and 2560-token prompts | startup warmup ladder | 11-benchmark-the-hardware-first |
| Throughput | (hand-launched server, nice 5) | macOS over non-interactive SSH | 28.5 tok/s prefill (89.8 s) on 2,560-token prompt; 25k-token prompt unfinished after 10 min; production median ~320 tok/s | — | 2,560-token prompt | nohup … & (nice 5) | 12-your-ssh-benchmark-runs-at-nice-5 |
| Throughput | (priority table — no throughput) | macOS | — (nice levels: nohup=ni 5/pri 31, screen=ni 0/pri 31, launchd=ni 0/pri 4) | — | — | launch method | 12-your-ssh-benchmark-runs-at-nice-5 |
| Memory / VRAM | 125B MoE (6B active, 512 experts, top-10 + 1 shared, 48 layers, hybrid linear attention) | 8-bit · 256 GB Mac Studio | — | 192.2 GB weights | native 262k context | — | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | 4-bit mixed-precision protected · "this chip" (256 GB Mac Studio) | ~30 tok/s peak | 111.7 GB | — | — | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | plain 4-bit · 256 GB Mac Studio | — | 111.6 GB | — | — | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | text-only 4-bit (speculative head kept) · 256 GB Mac Studio | — | 94.5 GB | — | — | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | 2-bit (exists, not run) | — | 68–80 GB | — | — | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | bf16 base | — | 360 GB (cannot fit 256 GB box) | — | do-not-convert-locally | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | vendored single-file workaround · "this exact chip" | ~385 tok/s prefill | 111 GB peak | — | reviewer-validated | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | 125B MoE (same) | — | speculative decoding 2.33–2.62× | KV cache ≈ 98 KB/token (2 KV heads × 256 dim × 48 layers); 128k context ≈ 13 GB | 128k context ≈ 13 GB KV | — | 24-a-192gb-model-on-a-256gb-box |
| Memory / VRAM | — (kernel panic cause) | 256 GB Mac Studio | — | wired GPU memory limit hand-raised to 229,376 MB = 224 GB of 256 (87.5% of RAM) | — | — | 24-a-192gb-model-on-a-256gb-box |
| Throughput | (4-bit matrix-vector kernel, single-matrix loop best-of-5) | 4-bit · Mac Studio (M3 Ultra) | 850–910 GB/s (cache-resident) | — | — | best of 5 after warm-up | 35-our-kernel-benchmark-measured-the-system-cache |
| Throughput | (4-bit matrix-vector kernel, cold) | 4-bit · Mac Studio (M3 Ultra) | ~600 GB/s | 768 MB distinct matrices per shape | — | round-robin [cold] | 35-our-kernel-benchmark-measured-the-system-cache |
| Throughput | (decode kernel retune) | Mac Studio (M3 Ultra) | +7% decode throughput, +7% speculative pass (bit-identical) | whole model 15.7 GB | — | 4 → 1 rows/simdgroup (decode), 2 (two-token variant) | 35-our-kernel-benchmark-measured-the-system-cache |
| Throughput | — (profiler + drift findings) | Mac Studio (M3 Ultra) | absolute throughput drifts ±20–30% on shared box | — | — | profiler ~200 µs sync floor | 35-our-kernel-benchmark-measured-the-system-cache |
| Failure rate / token budget | Benchmark: items returning empty text (no tool call) | — | 11 of 60 = 18.3%; 15 of 60 hit finish_reason=length | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Mean score on truncated vs. rest | — | 0.257 vs 0.944 | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Two scenarios burned full budget with zero output | — | 12,288 tokens | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Reasoning-on budget threshold for empty content: None | — | below ~3,200 tokens | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Reasoning off → yes/no accuracy collapse | — | 71.7% → 61.0%; "no" bucket 75% → 39% | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Production truncation over 30 days | — | 1 in 1,520 requests = 0.07% | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Max completion observed in production | — | 17,542 tokens | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Monitoring reported finish_reason=length (before fix) | — | 98.1% of requests | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Health check default / fix / result | — | max_tokens=16 → tried 256 → 1024; length share 98.1% → 24.6% | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Health-check reasoning span on "What's 1 + 1?" | — | 48 to 440 tokens | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Truncation alarm threshold vs. baseline | — | pages above 3% vs. 0.07% baseline | — | — | — | 04-benchmark-said-18-percent-production-said-0.07 |
| Failure rate / token budget | Compute deficit from wedged clock governor | — | 3.3× steady (SoC-wide; CPU ~2.4× slow); 3.2× recovery after reboot | — | — | — | 11-benchmark-the-hardware-first |
| Failure rate / token budget | Memory state ruling out swap | — | 57 GB free, 147 GB inactive, zero swap | — | — | — | 11-benchmark-the-hardware-first |
| Failure rate / token budget | Healthy/sick warmup threshold | — | ~350 tok/s at 2560 tokens healthy; under 150 tok/s = sick | — | — | — | 11-benchmark-the-hardware-first |
| Failure rate / token budget | Cache-benchmark overstatement | — | off by ~40% (850–910 vs ~600 GB/s) | — | — | — | 35-our-kernel-benchmark-measured-the-system-cache |
[ put the numbers to work ]
TURN BENCHMARKS INTO A DECISIONlocal AI, measured
These are the numbers behind every local-AI claim on this site. If you'd rather have a setup that hits them on your own hardware than read about how they were measured, start here.
book a free scoping call the $899 readiness auditScoping calls are free and no-obligation — bring your hardware and goal.