Ryzen AI Halo Review: AMD's DGX Spark And Mac Mini Challenger Tested
There are a handful of ways to evaluate AI performance on these small workstations, but we tend to favor experience over synthetic tests. Benchmarks like Geekbench AI or Procyon are useful for evaluating CPU or GPU performance on certain mathematic operations, but on these sort of things it makes more sense to just use the AI and jot down results. Ollama reports Time to First Token and Tokens per Second of whatever model is loaded. For RAG and document embeds, you can also use Ollama. Image generation done through Comfy Desktop can be timed, which is another easy data point.
Enter open-source benchmark with the creatively-bankrupt name Local AI Bench. Created and maintained by Ben Funk of HotHardware (hey, I know that guy!), Local AI Bench is an automated tool that installs everything you need from a single batch file on Windows or Bash script on Linux and macOS. It's compatible with AMD GPUs via ROCm, NVIDIA GPUs via CUDA, and Apple's MetalML API. It'll also fall back to CPU if it can't find anything else. Download it from the GitHub repository and ensure Python is at version 3.11 or later, and you're good to go.
"Small" LLM Performance on AMD Ryzen AI Halo
The LLM benchmarks are split into three size classes: small, medium, and large. Small benchmarks will probably run on GPUs with 16 GB or less of VRAM, like the Radeon RX 9070 XT or GeForce RTX 5070 Ti. They'll fit into the Ryzen AI Halo's memory space no sweat.The benchmarks probably bear some explanation, so let's start off easy with Llama 3.1 8B with a 3-bit quantization of the weights. First the results, then we'll talk about them. This is the lightest model we'll test, at least in terms of memory footprint.
For each model we have two charts: tokens per second and time to first token, for different prefill values. The prefill is a long prompt comprised of 8K, 32K, or 64K tokens. The magic of cloud LLMs is that they have hidden guardrails in the form of system prompt add-ons, which pushes the token count up. Additionally, retrieval-augmented generation (RAG) or document attachments also have to be tokenized. Think of the 64K prefill as a detailed query with documents attached.
We can see as the prefill goes up, the generation goes down a bit and the time to first token rises. No surprise there. We can also see that the Ryzen AI Halo and NVIDIA DGX Spark are neck and neck, another non-surprise given their similar memory bandwidth. And the Mac Studio (2025) runs away with whatever it can run because it has way more memory throughput, and that's what inference is based on.
Hopefully that all makes sense, and we can turbo through the rest of the charts without such detailed explanations. So let's go up to a 4-bit quant.
Stepping up to a larger quant lowers performance and increases TTFT slightly, but it's less than 10% across the board.
Next up is Qwen3-14B, which we'll do in 8-bit and 4-bit quantizations. These probably are a little on the big side for 8GB GPUs, for reference.
There's no 64K prefill here because every system choked on it, because it has a maximum context window of 32k. Anything bigger and it starts shuffling stuff in and out of the window, performance tanks, and we go from measuring TTFT in milliseconds to minutes. It was awful and a huge waste of time troubleshooting. Anyway....
Results are basically the same pecking order. The Ryzen AI Halo trails the DGX Spark but it's not by much. We'd be hard-pressed to tell the difference between the two if we were accessing them remotely and couldn't see the operating system or the machine.
"Medium" LLM Performance on AMD Ryzen Halo
Medium sized models are going to stretch out the most capable consumer GPUs, but they do fit in the VRAM of a GeForce RTX 5090, for example, and they even manage to run on a GeForce RTX 5080 or Radeon RX 9070 XT as long as you don't mind performance falling off a cliff since half the model is stored in system RAM.The models use for this next batch of tests are GPT-OSS-20B and Qwen3.6-35B-A3B.
With Qwen3.6, we see the natural order restored, in which the two 128 GB systems are roughly matched up in token generation, although this time it's AMD with a slight edge at the 64k prefill level. The DGX Spark does take a somewhat noticeable amount of time to get started, as well.
"Large" LLM Performance on AMD Ryzen AI Halo
Finally we're at the big dawg stage. Meta's Llama 3.1 70B and GPT OSS 120B happen to be models that both AMD and NVIDIA say we should use to witness the firepower of their respective battle stations. So let's see how it went.
Even with a 3-bit quantization, none of these systems is particularly adept with Llama 3.1 70B. The time to first token is tolerable across the board, but nobody will really enjoy using an LLM that outputs responses in the six-to-seven tokens per second range. What if we bump it up to Q4?
Technically the Mac Studio outperformed the Ryzen AI Halo and DGX Spark, but it choked on the 64k prefill. The benchmark has a timeout period of five minutes, and if a model can't start responding by then it cuts its losses and moves on. That's exactly what happened here -- the 64 GB combined with Ollama's courageous attempts at using the SSD as swap space finally couldn't keep up at all. So that's why it falls to the bottom of the chart -- one DNF is basically a disqualification.
Performance was somewhat worse on the Q4 version of Llama 3.1 70B on the NVIDIA and AMD systems, although in this case AMD came out ahead, ever so slightly.
The biggest of the big dawgs is GPT OSS 120B. Let's see how that goes.
Performance was wildly improved compared to Llama above. All three systems were pleasant to use. And since the Ryzen AI Halo and DGX Spark have more than adequate memory space, you could actually do something else at the same time. Time to first token was brisk and the generation rate was fast enough that we don't think most folks will feel too terribly impatient.
Embedding Performance on Ryzen AI Halo
For embedding, we used Ollama again, which is still using ROCm on Windows. This is measured in different batch sizes as sentences per second. It's each line or finished thought that get imported and embedded by a special embedding model in Ollama. And things got unpredictable.
You'll see another entry on our charts, because I was concerned that the Ryzen AI Halo had been misconfigured. The DGX Spark and Mac Studio can do 100+ sentences of embedding at a time, but the Ryzen AI Halo dropped to a mere 13 with a batch size of 32, and 44 sentences per second at batch size 128. Then it fails entirely on the biggest batch.
As a troubleshooting step I ran this on a fourth system, a Windows PC based on the Ryzen 7 9800X3D with 64 GB of DDR5-6400 CL32 memory and a GeForce RTX 5080 with 16 GB of VRAM. Performance was no different. It seems like there's some sort of bottleneck on Windows, and it further seems like that bottleneck is hardware-agnostic. It failed to pass muster on both AMD and NVIDIA GPUs. It might be time for a different model.
Image Generation on AMD Ryzen AI Halo
Finally, it's time to make some AI "art" with ComfyUI. For this we chose two relatively large and very popular models: Stable Diffusion XL and FLUX.1-schnell.
Perhaps unsurprisingly, image generation is impacted by the resolution. More surprising to us is how close everything is between the two models at 1024x1024, while 1536x1536 has some differences teased out. While the Ryzen AI Halo was the slowest of our three entries, we're not unhappy with three-second generation times.
Next up it's time to dig into thermals and acoustics before we wrap up with conclusions.
