Running 30B-parameter LLMs on a Ryzen iGPU

What this achieves: turn the integrated Radeon 880M/890M (RDNA 3.5, gfx1150) on a Ryzen AI 9 HX –class mini-PC (e.g. Minisforum N5 Pro) into a big-model LLM engine using its large UMA “VRAM” — running models far too big for a small discrete GPU — without leaving Proxmox. It’s done with an LXC (shared host kernel = near-native, no VFIO/VM overhead), so every other VM/CT keeps running.

Real result: a 33B model (27 GB) runs 100% on the iGPU at ~25 tok/s, 128K context, on a 96 GB box with ~48 GB carved as iGPU memory (ollama reports ~71 GB usable VRAM). Proxmox and all guests untouched.

Nature of the win: this is a capacity win (run bigger models) and a prefill win (prompt processing is ~6–7× a CPU), not a huge generation-speed win — iGPU token generation is memory-bandwidth-bound (~1.3× a CPU). MoE/hybrid models (e.g. Nemotron-H) generate far faster than dense models of the same size.

---

Benchmarks (this deployment — Radeon 890M, gfx1150 → gfx1100)

Fixed prompt, num_predict=160, num_ctx=2048, via the ollama API. Gen = generation, Prefill = prompt processing.

Model	Params	Gen tok/s	Prefill tok/s	Cold load
llama3.2	3B	34.5	477	14 s
granite3.3	8B	14.6	271	32 s
mistral-small	24B (dense)	5.4	129	91 s
nemotron3	33B (hybrid)	24.5	88	194 s

iGPU vs CPU (same model — granite3.3:8b):

	Generation	Prefill
iGPU (ROCm)	14.6 tok/s	267.9 tok/s
CPU-only	11.4 tok/s	40.2 tok/s
iGPU advantage	1.3×	6.7×

Two things the numbers make obvious: (1) architecture beats parameter count — the 33B hybrid Nemotron outruns the 24B dense Mistral-Small ~5× (far fewer active params/token); (2) the iGPU’s edge is prefill (6.7×, compute-bound) far more than generation (1.3×, memory-bandwidth-bound) — so it shines on long-context / RAG / big-prompt work and on running models that simply won’t fit on a small dGPU. Cold load scales with model size (disk→VRAM) and is a one-time cost per load — keep hot models resident with OLLAMA_KEEP_ALIVE.

---

Why it’s not obvious

On a fresh Proxmox setup the iGPU is usually bound to vfio-pci (parked for passthrough) and unused, and even after you free it, ollama silently runs CPU-only because (a) the stock ollama install often ships without a ROCm backend, and (b) gfx1150 isn’t in ollama’s bundled ROCm kernels. Three separate things have to line up.

Prerequisites

• Proxmox VE host, AMD Ryzen AI / Radeon 880M/890M iGPU, and iGPU memory allocated in BIOS (the “UMA”/“VGM” setting — that’s your VRAM ceiling).
• ollama running in an LXC (this guide assumes CT id <CTID>; adjust paths for a VM or bare metal).
• Root SSH on the host. A reboot is required (see step 2), so schedule it.

---

Step 0 — Back up (2 min)

D=/root/igpu-backup-$(date +%s); mkdir -p "$D"
cp -a /etc/modprobe.d "$D/"; cp -a /etc/default/grub /etc/kernel/cmdline "$D/" 2>/dev/null
cp -a /etc/pve/lxc/<CTID>.conf "$D/"

Rollback = restore these + update-initramfs -u + reboot.

Step 1 — Identify the iGPU

lspci -nnk | grep -iA3 'VGA\|Display'

Note the iGPU’s PCI address (e.g. c6:00.0) and device id (the 890M is 1002:150e). Confirm it’s currently Kernel driver in use: vfio-pci.

Step 2 — Free the iGPU for amdgpu, then reboot

Two edits (leave the kernel cmdline alone — editing only modprobe.d keeps boot risk low). Keep any discrete GPU ids in the vfio list if you pass one to a VM.

# a) remove the iGPU device id from the vfio-pci id list
sed -i 's/1002:150e,//' /etc/modprobe.d/vfio.conf        # adjust id/filename to your setup
# b) stop blacklisting amdgpu (keep nvidia/nouveau/radeon blacklisted if present)
sed -i '/^blacklist amdgpu$/d' /etc/modprobe.d/blacklist.conf
update-initramfs -u
reboot

A runtime rebind (unbind vfio + bind amdgpu without rebooting) does not cleanly init an APU iGPU — it wants amdgpu from boot. The reboot is required.

After reboot, verify:

lspci -nnks <c6:00.0>        # -> Kernel driver in use: amdgpu
ls /dev/dri/renderD128 /dev/kfd
dmesg | grep -i 'amdgpu.*VRAM'   # -> "VRAM: NNNNNM ... ready"
/opt/rocm*/bin/rocminfo | grep -i gfx   # -> gfx1150 as a GPU agent

Step 3 — Pass the iGPU into the (unprivileged) ollama LXC

PVE 8.2+ dev passthrough handles the unprivileged idmap for you. Use the render group gid on the host:

getent group render                     # e.g. render:x:993:
pct set <CTID> -dev0 /dev/dri/renderD128,gid=993 -dev1 /dev/kfd,gid=993
pct start <CTID>
pct exec <CTID> -- ls -l /dev/dri/renderD128 /dev/kfd   # both present, owned root:render

Ensure the ollama process is in the render group (it runs as root here, which works; otherwise add its user to render).

Step 4 — Give ollama a ROCm backend + the gfx override

Check what backends ollama actually has:

pct exec <CTID> -- ls /usr/local/lib/ollama/lib/ollama/   # look for a 'rocm' dir next to cuda_v*/libggml-cpu-*

If there’s no rocm dir, install the version-matched ROCm bundle from ollama’s GitHub release. Extract it into the SAME directory as the other backends — a very common mistake is landing it one level up, which ollama never scans (→ silent CPU fallback):

V=$(pct exec <CTID> -- /usr/local/bin/ollama --version | grep -oE '[0-9]+\.[0-9]+\.[0-9]+' | head -1)
pct exec <CTID> -- bash -c "cd /tmp && \
  curl -fsSL -o r.tzst https://github.com/ollama/ollama/releases/download/v$V/ollama-linux-amd64-rocm.tar.zst && \
  tar -C /usr/local --zstd -xf r.tzst && rm r.tzst"
# if it landed at /usr/local/lib/ollama/rocm but the backends live in .../lib/ollama/, move it:
pct exec <CTID> -- bash -c 'B=$(dirname $(find /usr/local -name libggml-cpu-*.so|head -1)); \
  H=$(dirname $(find /usr/local -name libggml-hip.so|grep -v "$B/rocm"|head -1)); \
  [ -n "$H" ] && [ "$H" != "$B/rocm" ] && mv "$H" "$B/rocm"'

Add the gfx override (maps gfx1150 → gfx1100, which ollama’s rocblas does ship) and restart:

pct exec <CTID> -- bash -c 'mkdir -p /etc/systemd/system/ollama.service.d; \
  printf "[Service]\nEnvironment=\"HSA_OVERRIDE_GFX_VERSION=11.0.0\"\n" > /etc/systemd/system/ollama.service.d/rocm.conf; \
  systemctl daemon-reload && systemctl restart ollama'

Step 5 — Verify + test

pct exec <CTID> -- journalctl -u ollama | grep 'inference compute' | tail -1
#  -> library=ROCm compute=gfx1100 ... type=iGPU total="~71 GiB"
pct exec <CTID> -- /usr/local/bin/ollama run <a-30B-model> "hello" --verbose
pct exec <CTID> -- /usr/local/bin/ollama ps          # -> PROCESSOR: 100% GPU

---

Gotchas that cost the most time

1. ROCm bundle path — must sit beside the cuda_v*/libggml-cpu-*.so backends, not one dir up. Wrong path = ollama silently stays on CPU with no error.
2. HSA_OVERRIDE_GFX_VERSION=11.0.0 is required — ollama’s bundled rocblas has no native gfx1150 kernels.
3. Reboot required — the APU iGPU won’t cleanly bind amdgpu via a runtime rebind.
4. Unprivileged LXC — use PVE dev0/dev1 ,gid=<render> (do not hand-roll lxc.cgroup2/idmap); put ollama in the render group.
5. Perf expectations — capacity + prefill win, modest generation speedup vs CPU. Prefer MoE/hybrid models for speed at large sizes. First load of a big model is disk-bound (one-time); keep hot models resident with OLLAMA_KEEP_ALIVE.

Rollback

Restore /etc/modprobe.d/* (re-add the id to vfio.conf, re-add blacklist amdgpu), update-initramfs -u, reboot; pct set <CTID> -delete dev0,dev1.

---

Written from a working deployment on a Minisforum N5 Pro (Ryzen AI 9 HX PRO 370 / Radeon 890M, 96 GB, ~48 GB UMA), Proxmox VE 9.x, ollama 0.22 + ROCm 7.2. Adjust ids/paths/versions to your hardware.