Sizing a Local-AI Box for a 5-to-20-Person Office

Headcount is the wrong variable — concurrency is. How to size memory and throughput for an office that shares one machine, what fits in what, and where the real bottlenecks are.

Summary

The mistake almost everyone makes when sizing a shared AI machine is starting from headcount. A 20-person office is not twenty simultaneous users; most of the time it is one or two, with a lunchtime spike. What actually sets the hardware is concurrency — how many people hit the model at the same instant — and that number is driven by workflow, not payroll. Once you size for concurrency instead of headcount, the answer is usually more modest than expected: a single workstation-class box serves a small office comfortably, and the real constraint is memory bandwidth and total unified memory, not a rack of GPUs. This post lays out the sizing arithmetic, with every estimate labeled.

Headcount is the wrong variable

Think about how a small office actually uses a shared model. A paralegal drafts a letter. A dental hygienist writes a summary. A staff accountant extracts figures from a document. These are short, bursty interactions — someone types a request, gets an answer in a few seconds, and reads it for a minute. The machine is idle in between.

The result: a 20-person office rarely presents more than two or three simultaneous requests, and often zero. Peak concurrency is what matters, and the way to estimate it is to watch your own workflow — who uses the tool, how often, and whether those uses cluster. Concurrency of two is a very different machine than concurrency of ten, and the difference is not a factor of five in price, it is the difference between one box and a cluster.

The memory budget

For a single model serving a handful of users, total unified memory is the binding constraint, and the budget has three parts:

1. The model weights. A model's memory footprint is set by its parameter count and quantization. A 7–8B model at 4-bit quantization is roughly 4–5 GB of weights; a 30B-class model is in the high teens to low twenties of GB; a 70B-class model at 4-bit is around 40 GB. (The exact relationship is covered on the how-much-RAM guide.)
2. The KV cache per concurrent user. Every active request holds a key-value cache proportional to its context length. This is the concurrency term — two simultaneous long-context users cost roughly twice the KV cache of one. Long documents and long conversations inflate this fast.
3. Operating system and headroom. The OS, the page cache, and a safety margin so the machine doesn't swap.

The practical arithmetic: take the weights, add the KV cache for your realistic peak concurrency, add headroom, and buy the next memory tier up. The RAM calculator does this arithmetic for you; the model-fit tool checks whether a specific model fits a specific machine.

What fits in what

As a rough, label-as-estimate guide:

• 32 GB runs a 7–8B model comfortably for a handful of users, and is the budget baseline.
• 64 GB runs a 13–30B-class model with room for several concurrent users, which is the sweet spot for most small offices.
• 96 GB and up buys comfort: a bigger model, longer contexts, and more simultaneous users without tuning.
• 128–256 GB is where you go for the largest open models and genuinely many concurrent users — and where a single workstation like the Mac Studio (up to 512 GB unified memory) replaces a server room.

The single most important number to internalize is that these are shared machines. A 20-person office does not need twenty machines or a data-center GPU. It needs one machine with enough memory to hold the model and enough memory bandwidth to keep a few concurrent users at interactive speed.

Throughput, not just capacity

Memory decides whether the model fits. Memory bandwidth decides how fast it answers, and for an office that's the difference between a tool people use and a tool people give up on. Decode speed for a local model is, to a first approximation, bandwidth divided by bytes per token — which is why this matters more than raw compute for interactive serving.

Two published numbers set the range. The M3 Ultra's memory bandwidth is 800 GB/s; NVIDIA's DGX Spark delivers 273 GB/s on its GB10 chip. All else equal, the higher-bandwidth machine serves more tokens per second to each concurrent user, or serves the same speed to more users. For a 5-to-20-person office the M3 Ultra's headroom is usually overkill for the model size and exactly right for making a few simultaneous users feel instant.

The decision, in table form

The sizing decision collapses to two questions: how big a model do your tasks need, and what is your realistic peak concurrency.

• Solo or two-to-three light users, small model: 32 GB is enough; don't overspend.
• Small office, steady use, 7–13B model: 64 GB is the value point.
• Office that wants a larger model or longer documents: 96 GB.
• Larger model, many concurrent users, or regulated workloads with audit logging running alongside inference: 128–256 GB on a single workstation.

Whatever the tier, the two failure modes to avoid are buying too little memory (the machine swaps, and performance collapses — a shared box that thrashes is worse than no box) and buying too much compute for a workload that is memory-bound, not compute-bound. This is exactly the judgment the on-site audit exists to make, and why it is cheaper to scope correctly than to buy by guess.

The caveats that matter

Three honest caveats before you spend:

Context length changes the math. Long documents and long conversations blow up the KV cache, which is the concurrency term. If your office does long-document work, size memory for the context, not just the weights.

One slow user hurts everyone. On a shared box, a single request with a huge context can stall the others. This is a scheduling and configuration problem, not a hardware problem, but it is part of why a shared office deployment is more than "install and walk away" — see the setup process.

These are estimates, not measurements. The arithmetic here is the standard capacity-planning math, and it is labeled as such. The right numbers for your office come from your actual workflows, which is the first thing an audit establishes.

Key takeaways

1. Concurrency, not headcount, sets the hardware. A 20-person office is one or two simultaneous users most of the time; size for the peak, not the payroll.
2. Memory is the binding constraint. Weights plus per-user KV cache plus headroom — use the RAM calculator rather than guessing.
3. 64 GB is the value point for most small offices. 32 GB handles a small model and light use; 96 GB and up buys a bigger model and more simultaneous users.
4. Bandwidth decides whether it feels instant. The M3 Ultra's 800 GB/s versus the DGX Spark's 273 GB/s is the difference between a tool people use and one they abandon.
5. One machine, not a rack. A shared workstation serves a small office; you don't need a server room.
6. Context length and a single slow user change everything. Size for long documents, and treat a shared office box as a project, not an appliance.

Sources

• Apple Newsroom — M3 Ultra announcement (800 GB/s memory bandwidth): https://www.apple.com/newsroom/2025/03/apple-reveals-m3-ultra-taking-apple-silicon-to-a-new-extreme/
• Apple Newsroom — Mac Studio announcement (up to 512 GB unified memory): https://www.apple.com/newsroom/2026/09/the-new-mac-mini-and-mac-studio-are-available-today/
• NVIDIA — DGX Spark product page (GB10 chip, 273 GB/s): https://www.nvidia.com/en-us/products/workstations/dgx-spark/

The weight-footprint figures (a 7–8B model at 4-bit ≈ 4–5 GB, a 30B-class ≈ high-teens-to-low-20s GB, a 70B at 4-bit ≈ 40 GB) are rule-of-thumb estimates, not measurements — verify against the specific model's card before committing to a purchase.