← All guides Qalarc
On this page
Qalarc Guides · Local AI

Running Local AI on a Mini PC

2026-11-12 studio #local-ai#hardware#tools ~8 min read
The hardware tiers that actually matter, what runs at each one, and the 30-minute Ollama + ComfyUI setup — with measured numbers from our 128GB Strix Halo box instead of vendor marketing. By the end you'll have chat, image generation, video and voice running fully offline on a box the size of a lunchbox.
Affiliate link: the GMKtec hardware link below is an affiliate link — we earn a small commission at no extra cost to you. It funds the labs on this site, and we only link hardware we run daily.

Why a mini PC, and why local

One quiet box on the desk, everything offline, zero marginal cost per generation. That's the whole pitch. The 2026 version of this guide is short because unified memory changed the math: a modern mini PC with 64–128GB of shared RAM runs 30B-parameter MoE language models and 14B video models that used to demand a multi-thousand-dollar discrete-GPU rig[1].

Local wins on volume, privacy, custom characters and cost-per-generation at scale. Local loses on frontier-class quality and occasional use — a few images a month is still cheaper on cloud services. We run local because the studio generates thousands of items a week[2].

The hardware tiers

Three tiers cover everything. The spec that matters is unified memory bandwidth and capacity — not CPU cores, not "NPU TOPS".

TierMemoryWhat runsHonest verdict
Entry (~$400–800)16–32GB7–14B dense LLMs, SD-class images, TTSFine for chat + images; skip video
Mid (~$1,000–1,600)64GB30B MoE LLMs, FLUX images, 5B videoThe sweet spot for most people
Flagship (~$1,800–2,200)128GB unified30B MoE at 48+ tok/s, FLUX, 14B video, full audio studioOne box does everything; what we run

Our flagship reference is the GMKtec EVO-X2 AI — AMD Ryzen AI Max+ 395 (16 Zen 5 cores) with the Radeon 8060S iGPU on 256 GB/s of LPDDR5X-8000 unified memory, 128GB. It's the machine behind every benchmark on this site[3]:

GMKtec EVO-X2 AI — launch discounts (AWIN) →

The one spec to read carefully: memory bandwidth. 256 GB/s moves a 19GB MoE model ~13 passes/second on paper — and MoE models only use ~3B active parameters per token, which is why they hit 48–52 tok/s here while dense models crawl[4]. Buy bandwidth, run MoE.

What actually runs (measured on our box)

From the Local Model Atlas — everything below is installed and in daily studio use, not a listicle:

WorkloadModelDiskMeasured
Chat / agents (daily driver)GLM 4.7 Flash 30B-A3B MoE19GB48 tok/s on 128GB
General + codingQwen3.8 27B17GB~11 tok/s — usable, patient
Fast imagesz-Image Turbo12GB~26s per 1216×832
Best imagesFLUX.1 Krea-dev fp812GBSlower, better composition
Text-to-videoWan 2.2 ti2v 5B9.4GBThe practical video entry point
Voice cloningF5-TTSsmallBest quality-per-ease we've used
MusicACE-Step~7GBFull tracks from a prompt

The full stack — LLMs, image/video, audio — is roughly 200GB of model downloads and wants a 1–2TB NVMe. Budget the disk before the machine.

The 30-minute setup

1 · Ollama for text (10 min)

curl -fsSL https://ollama.com/install.sh | sh
ollama pull glm4.7-flash      # or the MoE model you chose
ollama run glm4.7-flash

On AMD unified-memory boxes, Ollama's ROCm build picks the iGPU up automatically. If a model runs but slowly with low GPU usage, your VRAM carve is too small — see step 4.

2 · ComfyUI for images + video (15 min)

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI && pip install -r requirements.txt
python main.py --listen 127.0.0.1:8188

Drop your checkpoints into models/checkpoints and load a workflow from the browser UI. The z-Image Turbo workflows are our house generator — every thumbnail on qalarc.com came out of it.

3 · Audio (5 min)

F5-TTS for voice cloning from a 10-second sample; ACE-Step for music; MelBand Roformer for stem separation. Each is a small pip install — the audio models are light next to the video ones.

4 · Tune the memory carve (AMD unified memory)

On Strix Halo boxes, set the UMA carve in BIOS to 32GB: a Q4 30B-class model fits fully in "VRAM" (100% GPU residency, measured) while the OS keeps the rest. Bigger models spill into GTT/unified automatically — slower but it works.

Re-benchmark after every runtime update. One Ollama upgrade alone moved GLM 4.7 Flash from ~38 to 48 tok/s on our box. Runtimes move fast; yesterday's numbers are stale.

The measured reality (including the correction)

We publish our own correction openly: early bandwidth math suggested 25–30 tok/s for dense 70B models on this hardware. Reality is 3–5 tok/s. Strix Halo is a MoE and mid-size platform, not a 70B platform — don't buy it for dense 70B, no matter what the marketing implies[4].

When NOT to go local

Occasional use (a few images a month), frontier-class quality demands, or zero-admin requirements → cloud APIs are still the right answer. Local wins on volume, privacy, custom characters and offline operation. For us, local also powers the games and labs — the splitscreen guide's reference machine is this same box pulling double duty as a couch console.

Notes & sources

  1. Unified-memory platforms (AMD Strix Halo, Apple Silicon) changed the capacity math: "VRAM" and system RAM are the same pool, so 14B video models fit where discrete 24GB cards can't. ↩
  2. The full measured catalogue lives in our Local Model Atlas — 19 LLMs plus the image/video/audio stack with real disk sizes and speeds. ↩
  3. Platform deep-dive with the full comparison table vs DGX Spark and Mac Studio: Strix Halo for local AI — measured reality. ↩
  4. The correction is documented in the Strix Halo deep-dive: bandwidth arithmetic gives the ceiling; architecture decides where you land under it. ↩

Discussion …

Footnote numbers above link here — corrections, better methods and "actually this broke for me" are all welcome.

Ask the Qalarc guide