./gear/old-gaming-pc-local-ai — buyers' notes

Turn an old gaming PC into a local AI box

An old gaming PC is already most of a local AI box. VRAM is the only spec that decides what fits: 8GB runs 7–8B at q4, 24GB runs 13–14B at q8 or 30–34B at q4. Keep the model fully in VRAM, and the used 24GB card is the classic upgrade.

You already own most of the machine. Here's what your VRAM actually runs, why a used 24GB card is the classic move, and the walkthrough from bare tower to serving a model.

[ the premise ]

A gaming PC is a local AI PC with a different sticker on the case. It already has the power supply, the case, the CPU, the system RAM, a fast disk, and — most importantly — a graphics card with real VRAM. The cloud bill you're trying to escape runs on the same silicon. So the weekend project isn't "build a machine"; it's "decide whether the machine you already own fits a model you'd actually use." Two questions answer that: how much VRAM is on the card, and can you live with the result at that size? Everything after that is a few hours of software.

[ the VRAM ladder ]

VRAM is the only spec that decides what fits. Weights cost roughly 0.6 bytes per parameter at q4 and ~1.1 at q8; add a gigabyte or three for context (the KV cache) and runtime overhead. Read down to your card:

The pattern to notice: each step up buys you either more context at the same size or the next model class up — not both. Pick which one you care about before you spend.

[ fit first: quantization beats offload ]

The single most important rule in local AI: a smaller model that fits in VRAM beats a bigger one that spills into system RAM. When every layer is on the card, the GPU reads weights at its own memory bandwidth — well over 1TB/s on anything modern. When you offload a few layers to system RAM because the model doesn't fit, the GPU has to reach across the PCIe bus every time those layers run. PCIe 4.0 x16 moves roughly 32GB/s in each direction, and fast DDR5 system RAM is a fraction of VRAM bandwidth — the hop across the bus is the bottleneck. It isn't a compute problem; the card has compute to spare. It's a bandwidth problem, and bandwidth is the one thing you can't buy back with a better CPU.

The fix is counterintuitive but simple: drop a quantization level or a model size until everything fits. A 7B at q4 that runs fully on the card will feel dramatically faster than a 13B at q4 that's 80% on the card and 20% offloaded. "It loads" and "it's pleasant to use" are different sentences.

[ the CPU-only case ]

You can skip the GPU entirely and run models on the CPU with system RAM. It works. It is also slow — small models creep along at a reading pace or worse, and the gap only widens as models grow. But there are three situations where it's genuinely the right call:

CPU-only is the free way to answer the real question — "is local AI useful to me?" — before you spend anything on hardware.

[ the used 24GB card ]

If your current card can't reach the sweet spot, the classic local-AI move is a used 24GB card. Why 24GB specifically: it's the smallest size that runs a 30B-class model at q4 fully in VRAM, and it's where the price-per-gigabyte of used VRAM tends to be best — check current prices rather than trusting a number here, because the used market moves weekly. Why used: new high-VRAM cards are priced for people who need them for work; buying one just to chat with a local model is hard to justify, and the used market is where the value actually lives.

The risks are real, and they come in categories rather than specific cards:

None of this makes used a bad idea — it makes it a researched idea. Buy the seller as much as the card.

[ Linux vs Windows, CUDA vs ROCm ]

The operating-system decision is really a driver decision. NVIDIA on Linux is the least-friction path: install the proprietary driver, install the CUDA toolkit, and Ollama or llama.cpp just work. NVIDIA on Windows is also fine — native drivers are solid, and WSL2 gives you the Linux toolchain with the Windows driver doing the CUDA work underneath. AMD is where it gets honest: ROCm is real and improving, but it's officially supported on a narrower list of cards and distributions, and it costs weekends you won't get back. If your old card is AMD and it's not on the supported list, treat it as CPU-only or sell it toward a used NVIDIA card.

[ the walkthrough: Ollama ]

Ollama is the fastest route from a bare tower to a running model. On Linux:

curl -fsSL https://ollama.com/install.sh | sh

Then pull a model and talk to it (swap the tag for whichever open model you prefer):

ollama pull llama3.2:3b
ollama run llama3.2:3b

Ollama already runs as a server on localhost:11434, so any client that speaks the OpenAI-style API can point at it. To expose it on your home network, or to keep models off a small system disk, set the two environment variables that matter before starting it:

export OLLAMA_HOST=0.0.0.0:11434
export OLLAMA_MODELS=/mnt/bigdisk/ollama
ollama serve

That second line is the one people miss: models are big, and they live in ~/.ollama/models by default. Point OLLAMA_MODELS at a big disk before you pull a dozen models, or you'll be surprised where the space went.

[ the walkthrough: llama.cpp ]

If you want to be closer to the metal — or you're on a card Ollama's defaults don't optimize for — llama.cpp is the engine underneath. Build it for CUDA:

git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
cmake -B build -DGGML_CUDA=ON
cmake --build build --config Release -j

(On an AMD card that's on ROCm's supported list, swap -DGGML_CUDA=ON for -DGGML_HIP=ON.) Then download a GGUF file from a quantized model repo on Hugging Face and run it:

./build/bin/llama-cli -m ~/models/model-Q4_K_M.gguf -p "Write a haiku about a dusty GPU." -ngl 999

-ngl 999 offloads every layer to the GPU — the fit-first rule in one flag. -ngl 0 runs fully on CPU. Everything between the two is the offload trap covered above; when in doubt, pick a quantization that lets you set it to 999.

[ power, thermals, PSU — the checklist ]

[ honest limits ]

Be clear-eyed about what this box is and isn't. A repurposed gaming PC will not run 70B-class models well — that's a multi-GPU build or a big unified-memory machine, not a weekend on an old tower. It will not serve a team at low latency while it's busy doing something else. And it will not be quiet, power-sipping, or zero-maintenance the way a dedicated appliance is. It will run a 7B–13B assistant — or a 30B at q4 on a 24GB card — entirely locally, privately, and free of per-seat fees. Which is most of what people actually want.

The stopping rule: stop upgrading the old PC the moment the next upgrade costs more than a used 24GB card, or the moment you realize you want 30B+ models with zero driver work. At that point the honest answer is a Mac Studio with unified memory or a fresh build — not another weekend coaxing an aging tower. The fork is laid out in Mac Studio vs GPU PC, and if you're starting from zero rather than an old box, the under-$3,000 builds are the shopping list.

RELATED NOTES Mac Studio vs GPU PC — the honest comparison → · RAM vs VRAM — the full memory math → · AMD vs Mac for inference → · three working builds under $3,000 →

CHECK PRICES see options at the retailer →

AFFILIATE DISCLOSURE: links in this post are affiliate links — if you buy through them I earn a small commission at no extra cost to you. RobotNurse is independent; I only recommend hardware I would actually deploy. No vendor paid for placement.