AI-narrated version of this post using a synthetic voice. Great for accessibility or listening while busy.
If you have tried to run a 70-billion-parameter model at home, you already know the wall you hit. Most consumer graphics cards carry somewhere between 8GB and 16GB of VRAM, and even flagships top out at 32GB, and the moment a model spills past that, generation speed collapses as layers get shuffled across the PCIe bus. The usual fix is more cards, more power draw and a case big enough to hold it all.
AMD’s Strix Halo platform, sold as the Ryzen AI Max+ 395, takes a different route. The CPU, the 40-compute-unit Radeon 8060S iGPU and the XDNA 2 NPU all share a single pool of LPDDR5X soldered to the board, and on 128GB configurations a very large slice of that can be handed to the GPU as VRAM. That changes which models fit on a machine the size of a thick hardback book.
This guide is for anyone deciding whether one of these mini PCs is the right host for Ollama, llama.cpp or an inference container at home. We compared the current Strix Halo mini PCs on memory, cooling and I/O, and we look honestly at where a used graphics card or an Apple Silicon machine still makes more sense.
What unified memory actually buys you
The headline benefit is capacity, not raw speed. A 70B model quantised to roughly 4 bits per weight needs somewhere in the region of 40GB for weights alone, plus room for the KV cache, which grows with context length. On a 128GB Strix Halo box that fits comfortably in the GPU’s allocation with space left over for the operating system and whatever else you are running.
BOSGAME lists up to 96GB of dynamic allocation to the iGPU on the M5, which gives you a sense of the ceiling. MINISFORUM describes the same idea as a shared, low-latency pool that removes the copy step between system RAM and dedicated VRAM. In practice that means loading a model is a matter of reading it off the SSD, not reading it off the SSD and then pushing it across a bus into a separate memory space.
The second benefit is mundane but real: one box, one power brick, no riser cables. For an always-on home inference server that sits next to a Proxmox home lab node rather than in a tower under the desk, that compactness matters.
- ▶ FLAGSHIP AMD RYZEN AI MAX+ 395 MINI PC – Packing 16 Zen 5 cores, 32 threads (via SMT), 64MB L3 cache, and a 5.1GHz…
- ▶ AMD RADEON 8060S iGPU – DESKTOP‑GRADE GAMING & CREATION – No discrete GPU needed. With 40 RDNA 3.5 compute units an…
- ▶ 128GB LPDDR5X‑8000MHz + 2TB SSD + DUAL M.2 SLOTS – Onboard 128GB memory at 8000MHz offers 45% more bandwidth than L…
Memory bandwidth is the real ceiling
Token generation on a dense transformer is memory-bound. For every token produced, the engine has to stream essentially every active weight out of memory and through the compute units, so your tokens per second scales with bandwidth far more than with FLOPS. This is why the question “how fast is the iGPU” is the wrong question.
Strix Halo runs a 256-bit LPDDR5X interface at 8000MT/s, which GMKtec describes as eight-channel. That is a large step up from the dual-channel SODIMM memory in an ordinary mini PC, and it is the entire reason this platform can run big models at usable speed. It is still well short of the bandwidth on a modern discrete GPU’s GDDR or HBM stack.
The practical consequence: a 70B model will load and run, but it will feel deliberate rather than instant. Smaller models in the 7B to 32B range are where these machines feel genuinely responsive, and mixture-of-experts models do unusually well because only a fraction of the weights are active per token. Prompt processing, which is compute-bound rather than bandwidth-bound, is a separate matter and benefits from the 40 RDNA 3.5 CUs.
Context length eats memory too
Long context windows inflate the KV cache quickly, and on a 48GB card that is often what forces a compromise. With 128GB on board you can be generous with context instead of trimming it to fit, which for document analysis or long agent sessions is worth more than a few extra tokens per second.
Software: ROCm, Vulkan and what works today
Getting an AMD iGPU to accelerate inference is less automatic than plugging in an NVIDIA card. Two paths dominate. ROCm is AMD’s CUDA equivalent and gives the best performance when the version, kernel and driver line up; support for Strix Halo’s gfx1151 target arrived later than enthusiasts wanted, so check that your distribution ships a recent ROCm before committing to a workflow.
The Vulkan backend in llama.cpp is the pragmatic alternative. It is less fussy about driver versions, works on Windows and Linux, and for many quantised models lands close enough to ROCm that the difference is academic. Ollama builds on llama.cpp, so if you are following our walkthrough on self-hosting an AI chatbot with Ollama and Open WebUI, you inherit whichever backend your build was compiled against.
One thing to set expectations on: the 50 TOPS XDNA 2 NPU is not what runs your LLM. NPUs target small, quantised, low-power workloads and are addressed through separate toolchains. We cover that distinction in more detail in our look at Copilot+ PCs and NPU mini PCs for local AI. Treat the TOPS figure as a specification, not as your inference engine.
- 🔥【Powerful Performance & AI Voice】Beelink GTR9 Pro mini pc equips with AMD Processor Ryzen AI MAX+ 395(16C/32T, up t…
- 🔥【Lastest GPU 8060S & XDNA 2 NPU】Built on the RDNA 3.5 architecture, the AMD Radeon 8060S Graphics iGPU features 40…
- 🔥【Large Capacity Storage & Quiet】The AI Mini PC comes with 128GB(16GB x8) LPDDR5X high frequency memory, which can d…
Comparing the current Strix Halo mini PCs
All four of these use the same APU, so differentiation comes from cooling design, networking and expansion. The table below uses manufacturer-stated specifications only.
| Model | Memory (stated) | Networking | Expansion | Notable |
|---|---|---|---|---|
| GMKtec EVO-X2 | Eight-channel LPDDR5X up to 8000MT/s | 2.5GbE, Wi-Fi 7, BT 5.4 | Dual USB4 40Gbps | Quad-display 4K/8K output via HDMI 2.1 and DP 1.4 |
| Beelink GTR9 Pro | 128GB (16GB x8) LPDDR5X | Dual 10GbE, Wi-Fi 7, BT 5.4 | Dual M.2 PCIe 4.0, up to 16TB total | DP 2.1 8K60, dual speakers, lifetime technical support |
| MINISFORUM MS-S1 MAX | 128GB LPDDR5X-8000MT/s unified | Dual 10GbE, Wi-Fi 7 | Full-length x16 slot (PCIe 4.0 x4 electrical), dual M.2 (x4 + x1) up to 16TB RAID 0/1 | Six heat pipes, dual turbine fans, 130W continuous power, 320W PSU |
| BOSGAME M5 | 128GB LPDDR5X-8000MT/s, up to 96GB to iGPU | 2.5GbE, Wi-Fi 7, BT 5.4 | 2TB SSD included, second M.2 slot | Dual USB4 40Gbps, quad 8K60 display output |
The MINISFORUM stands out for two reasons. Its stated 130W continuous power target and six-heat-pipe, dual-fan cooling assembly suggest it is built for sustained rather than bursty load, and the full-length PCIe slot (x4 electrical) is unusual in this class. MINISFORUM also documents a two-unit cluster running a 235B Q4 model at 10.87 tok/s, which is a manufacturer figure but a useful indication of where this hardware sits.
- 【High-Performance APU】The MS-S1 MAX features an AMD Ryzen AI Max+ 395 APU, integrating a Zen 5 architecture CPU (up t…
- 【128GB UMA Memory】Equipped with up to 128GB of LPDDR5x-8000MT/s unified memory, it enables the CPU and GPU to access…
- 【Flexible Expansion】The MS-S1 MAX features USB4 V2 (up to 80Gbps), dual 10GbE LAN, HDMI 2.1 (up to 8K60), a full-leng…
Cooling and noise under sustained load
Inference is not a benchmark burst. A model answering a long prompt pins the iGPU for minutes at a time, and in a small chassis the fan curve is what determines whether the machine throttles. Expect any of these to be audible under full load; they are compact boxes pushing over a hundred watts through a single heatsink.
If quiet operation matters, two things help. Put the machine somewhere with genuine airflow and clearance rather than inside a cabinet, and consider capping the APU’s power target in firmware, which usually costs less performance than it saves in noise. Anything involving in-wall power runs or new circuits for a rack should follow local electrical code, with a licensed electrician where your jurisdiction requires one.
Who should choose something else
If your models fit in 24GB or 32GB, a discrete GPU will be considerably faster for the same money category, because bandwidth is the limit and GDDR wins that fight. Our round-up of high-VRAM GPUs for serious local AI is the better starting point there, and a used professional card with 48GB covers a lot of ground.
Apple Silicon is the closest philosophical rival, since it uses the same unified memory idea with comparable or higher bandwidth on the higher-end chips, plus a mature Metal backend in llama.cpp. The trade-off is flexibility: a mini PC runs Linux natively, takes standard NVMe drives and, in the MINISFORUM’s case, accepts a PCIe card.
There is also the option of a smaller mini PC plus external graphics over OCuLink, which we examine in our guide to eGPU docks for mini PCs. That path gives you real VRAM bandwidth at the cost of two boxes and a cable. And if you only need occasional access to very large models, renting compute remains the cheaper answer.
- EVOLUTION AMD RYZEN AI MAX+ 395 MINI PC – GMKtec EVO-X2 is the next evolution in AI mini PC Ryzen Strix Halo series….
- AI NPU with XDNA 2 ARCHITECTURE – Powered by 16 “Zen 5” CPU cores, 50+ peak AI TOPS XDNA 2 NPU and a truly massive in…
- AMD RADEON 8060S iGPU – The AMD Radeon 8060S offers all 40 CUs with up to 2.9 GHz graphics clock and uses the RDNA 3.5 architecture
Our recommendation
For a dedicated local inference box that will run for hours at a stretch, the MINISFORUM MS-S1 MAX is the one we would pick. Its stated continuous power handling, heavier cooling hardware and full-length PCIe slot (x4 electrical) make it the most future-proof of the group, and dual 10GbE suits a machine that sits alongside a NAS.
The Beelink GTR9 Pro is the better choice if you want dual 10GbE and two full-speed PCIe 4.0 M.2 slots in a smaller chassis, and the BOSGAME M5 arrives with a 2TB drive already fitted, which removes a step for anyone who just wants to pull models and start working. The GMKtec EVO-X2 leans hardest into display output and makes sense as a dual-purpose workstation.
Buy one of these because you need to hold a large model in memory on a small, quiet, single-box system. Buy a graphics card if raw tokens per second on a model that already fits is what you are optimising for.
How these picks were chosen: HomeNode compares products using manufacturer specifications, current Amazon listing details and published owner feedback. Products are not tested in-house, and listings change often, so confirm current specifications on the product page before buying.
Affiliate disclosure: As an Amazon Associate, HomeNode earns from qualifying purchases at no additional cost to you.