On this page
Running Local AI on a Mini PC
Why a mini PC, and why local
One quiet box on the desk, everything offline, zero marginal cost per generation. That's the whole pitch. The 2026 version of this guide is short because unified memory changed the math: a modern mini PC with 64–128GB of shared RAM runs 30B-parameter MoE language models and 14B video models that used to demand a multi-thousand-dollar discrete-GPU rig[1].
Local wins on volume, privacy, custom characters and cost-per-generation at scale. Local loses on frontier-class quality and occasional use — a few images a month is still cheaper on cloud services. We run local because the studio generates thousands of items a week[2].
The hardware tiers
Three tiers cover everything. The spec that matters is unified memory bandwidth and capacity — not CPU cores, not "NPU TOPS".
| Tier | Memory | What runs | Honest verdict |
|---|---|---|---|
| Entry (~$400–800) | 16–32GB | 7–14B dense LLMs, SD-class images, TTS | Fine for chat + images; skip video |
| Mid (~$1,000–1,600) | 64GB | 30B MoE LLMs, FLUX images, 5B video | The sweet spot for most people |
| Flagship (~$1,800–2,200) | 128GB unified | 30B MoE at 48+ tok/s, FLUX, 14B video, full audio studio | One box does everything; what we run |
Our flagship reference is the GMKtec EVO-X2 AI — AMD Ryzen AI Max+ 395 (16 Zen 5 cores) with the Radeon 8060S iGPU on 256 GB/s of LPDDR5X-8000 unified memory, 128GB. It's the machine behind every benchmark on this site[3]:
GMKtec EVO-X2 AI — launch discounts (AWIN) →
What actually runs (measured on our box)
From the Local Model Atlas — everything below is installed and in daily studio use, not a listicle:
| Workload | Model | Disk | Measured |
|---|---|---|---|
| Chat / agents (daily driver) | GLM 4.7 Flash 30B-A3B MoE | 19GB | 48 tok/s on 128GB |
| General + coding | Qwen3.8 27B | 17GB | ~11 tok/s — usable, patient |
| Fast images | z-Image Turbo | 12GB | ~26s per 1216×832 |
| Best images | FLUX.1 Krea-dev fp8 | 12GB | Slower, better composition |
| Text-to-video | Wan 2.2 ti2v 5B | 9.4GB | The practical video entry point |
| Voice cloning | F5-TTS | small | Best quality-per-ease we've used |
| Music | ACE-Step | ~7GB | Full tracks from a prompt |
The full stack — LLMs, image/video, audio — is roughly 200GB of model downloads and wants a 1–2TB NVMe. Budget the disk before the machine.
The 30-minute setup
1 · Ollama for text (10 min)
curl -fsSL https://ollama.com/install.sh | sh
ollama pull glm4.7-flash # or the MoE model you chose
ollama run glm4.7-flash
On AMD unified-memory boxes, Ollama's ROCm build picks the iGPU up automatically. If a model runs but slowly with low GPU usage, your VRAM carve is too small — see step 4.
2 · ComfyUI for images + video (15 min)
git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI && pip install -r requirements.txt
python main.py --listen 127.0.0.1:8188
Drop your checkpoints into models/checkpoints and load a workflow from the
browser UI. The z-Image Turbo workflows are our house generator — every thumbnail on
qalarc.com came out of it.
3 · Audio (5 min)
F5-TTS for voice cloning from a 10-second sample; ACE-Step for music; MelBand Roformer for stem separation. Each is a small pip install — the audio models are light next to the video ones.
4 · Tune the memory carve (AMD unified memory)
On Strix Halo boxes, set the UMA carve in BIOS to 32GB: a Q4 30B-class model fits fully in "VRAM" (100% GPU residency, measured) while the OS keeps the rest. Bigger models spill into GTT/unified automatically — slower but it works.
The measured reality (including the correction)
We publish our own correction openly: early bandwidth math suggested 25–30 tok/s for dense 70B models on this hardware. Reality is 3–5 tok/s. Strix Halo is a MoE and mid-size platform, not a 70B platform — don't buy it for dense 70B, no matter what the marketing implies[4].
- MoE 30B-class: 48–52 tok/s sustained — the whole reason to buy this tier.
- Mid dense ~27B: ~11 tok/s — usable with patience.
- Two concurrent MoE sessions: 34.3 tok/s each — parallel agents are nearly free.
- Prefill decays from 830 tok/s at short prompts to 217 tok/s at 25k — keep chat loops under ~8k context.
When NOT to go local
Occasional use (a few images a month), frontier-class quality demands, or zero-admin requirements → cloud APIs are still the right answer. Local wins on volume, privacy, custom characters and offline operation. For us, local also powers the games and labs — the splitscreen guide's reference machine is this same box pulling double duty as a couch console.
Notes & sources
- Unified-memory platforms (AMD Strix Halo, Apple Silicon) changed the capacity math: "VRAM" and system RAM are the same pool, so 14B video models fit where discrete 24GB cards can't. ↩
- The full measured catalogue lives in our Local Model Atlas — 19 LLMs plus the image/video/audio stack with real disk sizes and speeds. ↩
- Platform deep-dive with the full comparison table vs DGX Spark and Mac Studio: Strix Halo for local AI — measured reality. ↩
- The correction is documented in the Strix Halo deep-dive: bandwidth arithmetic gives the ceiling; architecture decides where you land under it. ↩
Discussion …
Footnote numbers above link here — corrections, better methods and "actually this broke for me" are all welcome.