![[H]ard|Forum](/styles/hardforum/xenforo/logo_dark.png){kind=logo}
- Joined
- Dec 19, 2005
- Messages
- 18,855
“Positioning, Technology & Architecture
AMD's Ryzen AI Halo Mini-PC is branded a "Developer Platform," but it really has the potential to be a personal AI appliance for power users running big models locally. Under the hood, this compact machine is built around the Ryzen AI Max+ 395 processor, codenamed "Strix Halo." It features a modern Zen 5 CPU with 16 cores plus SMT, for 32 threads. Also included is a RDNA 3.5 iGPU with 2560 shaders and an XDNA 2 NPU. This is one part of the magic; the other part is that there's 128 GB of fast LPDDR5X-8000 memory, which is more than in most consumer and enthusiast devices available. For AI, that capacity is everything: it decides whether a model fits at all, and this box holds big models that the vast majority of graphics cards can't touch. Normally you'd expect that such a machine would be a big tower, but thanks to its roots in the laptop space, Strix Halo fits into a small elegant box that you can leave on your desk as a work PC, or hide anywhere, out of the way, if you want it only for its compute capability. The successor, Gorgon Halo (Ryzen AI Max+ PRO 495), is already announced and can be configured with up to 192 GB of LPDDR5X-8533, but otherwise only slightly boosts clocks on the same CPU architecture and design.
Why not a 128 GB PC? Or a 5090?
Capacity isn't the whole story; it's both capacity and the bandwidth of that capacity. A desktop PC with 128 GB of DDR5 loads the same models, but typically the CPU reaches the data at only around 80-120 GB/s, which is a third of what this box offers. Also, you need a GPU that has access to the memory pool—a discrete PCI-Express graphics card won't work for huge models. That's also why we have the 32 GB RTX 5090 failing so many times in this review—it runs out of VRAM with large models and has to rely on additional memory from the main memory—over the slow PCIe bus. But let's be clear here: if your models fit in 32 GB, buy a GPU and never look back. Our testing shows that as long as the GPU memory doesn't overflow, it'll be MUCH faster. We also explored the "then I'll just run a model that fits into 32 GB" idea. Results from such AI models aren't even close to what modern large-class models put out. Do you need a 120B model for simple tasks? Absolutely not, but as you crank up the workload complexity you'll start seeing glaring differences. I also think this matters when deciding whether you'll use a local model for your task or a cloud-based option like ChatGPT or Claude, which run on multiple GPUs with terabytes of memory.
Who is it for
AI is still in its early stages, so there's still a lot of exploration happening, new models and developments appear almost daily; I wouldn't call the whole experience fire-and-forget. AMD did a fantastic job with the launch experience, the Ryzen AI Developer Center works extremely well. The web-based playbooks that show how to achieve various AI scenarios are also a great resource. While I felt like using them 1:1 as blueprints for my scenarios wasn't a perfect fit, they are an excellent source of inspiration and provide suggested paths to how to achieve your goals. I also like that the Ryzen AI Halo runs on Windows—NVIDIA's DGX Spark is Linux only, which creates an additional burden for Windows users—no matter how experienced they are—in getting results out of the machine. AMD does support Linux, too; actually the experience is really nice, you can plop in a spare SSD, grab the Linux install image from AMD's website, and be up and running in Linux within an hour—I tested it—it just worked. If your primary use of AI is search—that is, writing a question in a prompt to get an answer, even if that involves follow-ups, you're probably better off with Claude or ChatGPT. Things start to get more interesting as soon as you're starting to build more complex scenarios, and want to bring in external resources, possibly by calling external APIs and interfaces etc.
What I think is very important to highlight here is that the out-of-the-box path, using the Vulkan backend is definitely not the optimum setup. While that is compatible with all hardware and can get you results, you really want to switch to vLLM if somehow possible. Performance will be much better, like 8x as high, which turns this into a different kind of experience. The catch? On AMD, the vLLM code is still marked as experimental, while on NVIDIA it's battle-tested and production-ready. We still got it to work on AMD, and the results are good, so definitely explore that option if you can.
Privacy and Sovereignty
This is where a box like this really earns its place. Everything runs on the machine in front of you—your prompts, your documents, your code, your customers' data—and none of it ever leaves the machine. There is no cloud account, no logging your questions and answers, no provider that might train on your inputs, no terms of service that change overnight or aren't honored, no risk of data leaks (other than your own). You can pull the Ethernet cable and everything keeps working. If you're handling sensitive material: legal, medical, source code, personal data, you might want to consider a local-only approach, even if it means slower overall speed. Sovereignty is the other half: the models are open-weight and yours to keep—download them once, and they run forever, with no subscription, no rate limits, and no vendor retiring or "upgrading" the model you built your workflow around. This happened to me several times in the past. Anthropic rolled out a new Claude model and suddenly things that I knew were right a day ago required some extra love to fix. They definitely improved that in recent months, but there's no guarantee like a model file sitting on your drive. On the other hand, this is also a risk—you'll be spending $4K for hardware that will age over the years, while cloud providers will always have the latest and the greatest tech to run their models—not infinitely fast, but really well, so you keep paying, and their cost of entry is much lower. Setup is also not "free"—you'll be spending your time downloading stuff, getting it to run, maybe even compiling the latest beta branches from GitHub while a cloud-powered prompt just works, and has hundreds of engineers in the background that improve it all day.
NVIDIA DGX Spark?
This is the one real rival, and it's a powerful, competitive box. In terms of decode (how fast a response streams out), both are very similar, but when it comes to time-to-first-token (how long to wait for the start of the answer), the Spark is significantly faster. It's also much faster at image and video generation, because most AI development work is happening on NVIDIA, using CUDA, which is fully supported on the DGX Spark. NVIDIA's chip also has support for formats that are becoming increasingly important: FP8, FP4 and NvFP4, which offers the space-savings of 4-bit with only a small loss in precision. The catch? NVIDIA's 128 GB DGX Spark sells for $7,000 now after a recent price-increase, and even the new 64 GB model is $5,000—a thousand dollars more than this Halo for half the memory. One small thing worth mentioning here, too, is that the DGX Spark's fan turns off completely when idle, while the Ryzen AI Halo's fan runs all the time. It's quiet in idle, but you'll still hear it humming.
Final Words
Overall I have to say that I had a lot of fun with the AMD Ryzen AI Halo. Going beyond just cranking out a cookie-cutter Mini-PC review, but really exploring it in depth, to understand the product and tell you everything about it. I've always loved to come up with ideas for benchmarking, and figuring out custom scenarios for AI with our own data at TPU was a cool new experience. Last but not least, I also learned a great deal on the way, which is just as valuable in this coming Age of AI. I'll be looking at NVIDIA's DGX Spark in detail next, so stay tuned.“
Source: https://www.techpowerup.com/review/amd-ryzen-ai-max-395-strix-halo-mini-pc/3.html
AMD's Ryzen AI Halo Mini-PC is branded a "Developer Platform," but it really has the potential to be a personal AI appliance for power users running big models locally. Under the hood, this compact machine is built around the Ryzen AI Max+ 395 processor, codenamed "Strix Halo." It features a modern Zen 5 CPU with 16 cores plus SMT, for 32 threads. Also included is a RDNA 3.5 iGPU with 2560 shaders and an XDNA 2 NPU. This is one part of the magic; the other part is that there's 128 GB of fast LPDDR5X-8000 memory, which is more than in most consumer and enthusiast devices available. For AI, that capacity is everything: it decides whether a model fits at all, and this box holds big models that the vast majority of graphics cards can't touch. Normally you'd expect that such a machine would be a big tower, but thanks to its roots in the laptop space, Strix Halo fits into a small elegant box that you can leave on your desk as a work PC, or hide anywhere, out of the way, if you want it only for its compute capability. The successor, Gorgon Halo (Ryzen AI Max+ PRO 495), is already announced and can be configured with up to 192 GB of LPDDR5X-8533, but otherwise only slightly boosts clocks on the same CPU architecture and design.
Why not a 128 GB PC? Or a 5090?
Capacity isn't the whole story; it's both capacity and the bandwidth of that capacity. A desktop PC with 128 GB of DDR5 loads the same models, but typically the CPU reaches the data at only around 80-120 GB/s, which is a third of what this box offers. Also, you need a GPU that has access to the memory pool—a discrete PCI-Express graphics card won't work for huge models. That's also why we have the 32 GB RTX 5090 failing so many times in this review—it runs out of VRAM with large models and has to rely on additional memory from the main memory—over the slow PCIe bus. But let's be clear here: if your models fit in 32 GB, buy a GPU and never look back. Our testing shows that as long as the GPU memory doesn't overflow, it'll be MUCH faster. We also explored the "then I'll just run a model that fits into 32 GB" idea. Results from such AI models aren't even close to what modern large-class models put out. Do you need a 120B model for simple tasks? Absolutely not, but as you crank up the workload complexity you'll start seeing glaring differences. I also think this matters when deciding whether you'll use a local model for your task or a cloud-based option like ChatGPT or Claude, which run on multiple GPUs with terabytes of memory.
Who is it for
AI is still in its early stages, so there's still a lot of exploration happening, new models and developments appear almost daily; I wouldn't call the whole experience fire-and-forget. AMD did a fantastic job with the launch experience, the Ryzen AI Developer Center works extremely well. The web-based playbooks that show how to achieve various AI scenarios are also a great resource. While I felt like using them 1:1 as blueprints for my scenarios wasn't a perfect fit, they are an excellent source of inspiration and provide suggested paths to how to achieve your goals. I also like that the Ryzen AI Halo runs on Windows—NVIDIA's DGX Spark is Linux only, which creates an additional burden for Windows users—no matter how experienced they are—in getting results out of the machine. AMD does support Linux, too; actually the experience is really nice, you can plop in a spare SSD, grab the Linux install image from AMD's website, and be up and running in Linux within an hour—I tested it—it just worked. If your primary use of AI is search—that is, writing a question in a prompt to get an answer, even if that involves follow-ups, you're probably better off with Claude or ChatGPT. Things start to get more interesting as soon as you're starting to build more complex scenarios, and want to bring in external resources, possibly by calling external APIs and interfaces etc.
What I think is very important to highlight here is that the out-of-the-box path, using the Vulkan backend is definitely not the optimum setup. While that is compatible with all hardware and can get you results, you really want to switch to vLLM if somehow possible. Performance will be much better, like 8x as high, which turns this into a different kind of experience. The catch? On AMD, the vLLM code is still marked as experimental, while on NVIDIA it's battle-tested and production-ready. We still got it to work on AMD, and the results are good, so definitely explore that option if you can.
Privacy and Sovereignty
This is where a box like this really earns its place. Everything runs on the machine in front of you—your prompts, your documents, your code, your customers' data—and none of it ever leaves the machine. There is no cloud account, no logging your questions and answers, no provider that might train on your inputs, no terms of service that change overnight or aren't honored, no risk of data leaks (other than your own). You can pull the Ethernet cable and everything keeps working. If you're handling sensitive material: legal, medical, source code, personal data, you might want to consider a local-only approach, even if it means slower overall speed. Sovereignty is the other half: the models are open-weight and yours to keep—download them once, and they run forever, with no subscription, no rate limits, and no vendor retiring or "upgrading" the model you built your workflow around. This happened to me several times in the past. Anthropic rolled out a new Claude model and suddenly things that I knew were right a day ago required some extra love to fix. They definitely improved that in recent months, but there's no guarantee like a model file sitting on your drive. On the other hand, this is also a risk—you'll be spending $4K for hardware that will age over the years, while cloud providers will always have the latest and the greatest tech to run their models—not infinitely fast, but really well, so you keep paying, and their cost of entry is much lower. Setup is also not "free"—you'll be spending your time downloading stuff, getting it to run, maybe even compiling the latest beta branches from GitHub while a cloud-powered prompt just works, and has hundreds of engineers in the background that improve it all day.
NVIDIA DGX Spark?
This is the one real rival, and it's a powerful, competitive box. In terms of decode (how fast a response streams out), both are very similar, but when it comes to time-to-first-token (how long to wait for the start of the answer), the Spark is significantly faster. It's also much faster at image and video generation, because most AI development work is happening on NVIDIA, using CUDA, which is fully supported on the DGX Spark. NVIDIA's chip also has support for formats that are becoming increasingly important: FP8, FP4 and NvFP4, which offers the space-savings of 4-bit with only a small loss in precision. The catch? NVIDIA's 128 GB DGX Spark sells for $7,000 now after a recent price-increase, and even the new 64 GB model is $5,000—a thousand dollars more than this Halo for half the memory. One small thing worth mentioning here, too, is that the DGX Spark's fan turns off completely when idle, while the Ryzen AI Halo's fan runs all the time. It's quiet in idle, but you'll still hear it humming.
Final Words
Overall I have to say that I had a lot of fun with the AMD Ryzen AI Halo. Going beyond just cranking out a cookie-cutter Mini-PC review, but really exploring it in depth, to understand the product and tell you everything about it. I've always loved to come up with ideas for benchmarking, and figuring out custom scenarios for AI with our own data at TPU was a cool new experience. Last but not least, I also learned a great deal on the way, which is just as valuable in this coming Age of AI. I'll be looking at NVIDIA's DGX Spark in detail next, so stay tuned.“
Source: https://www.techpowerup.com/review/amd-ryzen-ai-max-395-strix-halo-mini-pc/3.html