./runbooks/apple-16gb — free runbook · apple silicon

16 GB of unified memory: one small model, kept honest.

A free public runbook grounded in two source files — our internal benchmark log and our deployment configs. It tells you what actually fits in 16 GB, what doesn't, and which numbers we really measured.

01 What fits — and what doesn't

Bottom line: on 16 GB you get a 14B-class model, 4-bit, with modest context — nothing bigger, and nothing we've speed-tested.

02 Throughput — not covered for 16 GB

Our only Apple-silicon throughput figures come from a 256 GB Mac Studio M3 Ultra. Listed for reference only — do not extrapolate them to 16 GB, which has far fewer GPU cores and far less memory bandwidth:

256 GB Mac Studio M3 Ultra (80-core GPU) — the only Apple hardware we measured:

  prefill 373.7 tok/s      25,970-token prompt             measured (posts/11)
  AEON 8-bit decode 42.9   dflash-mlx + DFlash2            measured (our configs)
  stock Qwen3.8 4-bit 66.9 dflash-mlx + DFlash2            measured (our configs)

16 GB Mac: no measured tok/s exists in our internal benchmark log or our deployment configs.  not covered

03 Memory headroom

16 GB is tight. macOS and your apps take several gigabytes before the model even loads, so the model weights are only part of the story. Our sources record the cautionary opposite: on the 256 GB Studio, hand-raising the wired GPU-memory limit to 224 GB — 87.5% of RAM — caused a kernel panic measured (our internal benchmark log). Lesson: never wire the ceiling; leave real headroom.

For qwen3:14b, keep context modest. The configs ran ollama_num_ctx 65536 on the mini measured config; on 16 GB a shorter context buys you more headroom for everything else estimate — we have no 16 GB KV-cache figures.

04 Recommended settings

Verbatim from our deployment configs — the real Ollama + Hermes wiring RobotNurse used:

# Ollama provider (Hermes)
model: qwen3:14b
provider: custom
base_url: http://10.0.0.2:11434/v1

# Hermes model block
model:
  context_length: 100000
  max_tokens: 2048
  ollama_num_ctx: 65536

05 What this page does not cover

SOURCES our internal benchmark log · our deployment configs · measured posts: benchmark the hardware first → · a 192GB model on a 256GB box →