Ran DS V4-Flash-0731 Locally on 3xMI50 32GB at 15 t/s: RADONZONE Magic or Hype?

I keep seeing posts about AMD GPUs hitting local LLM inference speeds that don’t entirely make sense. The latest one causing a stir: a r/LocalLLaMA user claiming DS V4-Flash-0731 runs locally on 3xMI50 32GB setups at roughly 15 t/s.

My immediate reaction was pure skepticism. We have been conditioned to think anything without an “NVIDIA” stencil requires a blood sacrifice to run. But I ran almost the exact same stack last weekend, and honestly? The numbers check out.

The Hardware Reality Check #

Let’s talk about the MI50. If you are not deep in the LocalLLaMA weeds, the MI50 is a 32GB Vega 20 datacenter GPU. You can grab these off eBay for around $200 a pop right now. Three of them gives you 96GB of VRAM for about $600.

That is aggressively cheap. It is also deeply annoying to set up.

NVIDIA gives you CUDA. AMD gives you ROCm, which is essentially a wild animal you have to slowly tame. Getting three MI50s running in sync without driver conflicts is a weekend project. But if you have the patience, these old datacenter cards are absolute units for memory bandwidth. The MI50 pushes 1 TB/s, which is why a 32GB card from 2018 can still hang with modern hardware for inference.

Hitting 15 t/s on DS V4-Flash-0731 #

DS V4-Flash-0731 is not a tiny model, but it is highly optimized. When you squeeze it into a 96GB VRAM pool across three MI50s, the math actually works.

The original poster was getting around 15 tokens per second for text generation. That speed is perfectly usable for interactive chat. It’s not lightning fast, but it feels responsive.

This speed is entirely a memory-bandwidth bottleneck. Because the MI50 has that 1 TB/s bandwidth, you can split the model across two or three cards using pipeline parallelism without the PCIe bus becoming a massive bottleneck.

The AMD Software Tax #

I love the MI50 for budget VRAM. It has one fatal flaw: the software stack will make you question your life choices.

NVIDIA’s ecosystem just works. You install the driver, you run your script, and it works. AMD requires you to read three forum posts, set six environment variables, and sacrifice a goat to the HIP runtime. The community is genuinely split on whether the savings are worth the headache.

I spent four hours last Tuesday fighting HIP errors just to get a standard quantized model running. If you are using Podman or Docker to containerize your ROCm setup, it is slightly easier, but you still have to pass the correct device mappings. If you are on bare metal, good luck.

Is This Overkill? #

Yes. This setup is absolute overkill for most people.

If you just want to run standard quantized 8B models, do not buy three MI50s. Buy a single 16GB RTX 4080 on the used market, or grab a Mac Studio with unified memory. The MI50 is loud, pulls 300 watts per card, and requires a massive power supply and a server case. Your electricity bill will reflect your life choices.

But if you want to poke at 70B class models locally? It is one of the cheapest ways to get there. Hetzner bare metal is obviously easier if you rent, but owning 96GB of VRAM outright for $600 is hard to beat.

I haven’t tested this specific rig on ARM yet, so your mileage may vary if you try to hack these into a modern Ampere workstation. Stick to x86 for this setup.

Ultimately, the real story here is that bottom-tier hardware can still punch way above its weight class. The MI50 is living proof that raw VRAM capacity still beats shiny new architectures when you need to load massive context windows. Just make sure you have a good pair of headphones to drown out the fan noise.