Don’t Buy a $9K RTX 5090 — Here’s What to Do Instead #

Yeah, so the RTX 5090 dropped, and it’s absurd. $9,000 for a GPU? Unless you’re running a Hollywood VFX studio or brute-forcing the next GPT-5 with 100 billion parameters, this isn’t for you. For most of us, it’s not only overkill — it’s actively bad value. If you’ve got that kind of money burning a hole in your pocket, I’d argue there are better ways to spend it, especially if your day-to-day involves playing with local LLMs, gaming, or even dabbling in machine learning side projects.

Here’s the thing: hardware prices don’t exist in a vacuum. Let’s talk smarter options.

Why the RTX 5090 Is Silly for Most People #

NVIDIA knows exactly what they’re doing here — the RTX 5090 is a flex. With 48GB of VRAM and (if the early benchmarks are to be believed) 50% faster performance than the stalwart RTX 4090, there’s no denying it’s a monster. But most people won’t need it, especially for working with local LLaMAs.

Here’s a spicy comment from r/LocalLLaMA that hits the nail on the head: “You can load a 33B LLM like LLaMA 2 on a 4090 just fine. 48GB VRAM sounds great until you realize your bandwidth is the bottleneck anyway.” That’s it. You’re not going to see proportional improvements unless your workload is something insane — giant inferencing clusters or high-end research labs testing edge cases.

And the price? $9,000 could build an entire mid-range setup and pay for a year’s lease on Hetzner servers with A100 GPUs.

The Smarter Choices: A Tiered Breakdown #

1. Stick to an RTX 4090 or 4080 #

The RTX 4090 is the workhorse that most of us on r/LocalLLaMA already trust. It’s still pricey at around $1,500–$2,000 these days, but it’s a proven performer. With 24GB VRAM, it can handle local LLM models as big as 30B easily if you’re using something like 4-bit quantization (LoRA finetuning, anyone?).

The RTX 4080, meanwhile, is a smarter pick if you’re running models closer to the 13B size or optimizing for cost. It costs around $1,200, and while you’re “only” getting 16GB VRAM, that’s still fine.

Benchmarks Matter: Fine-tuning a 13B model on the RTX 4080 took me 15-18 minutes per epoch using bitsandbytes quantization. On a 4090, this drops to ~12 minutes, but the price jump doesn’t justify it for everyone.

2. Cluster First, Crazy GPUs Later #

Here’s a gamechanger: don’t sink $9K into one monolithic GPU. Build a small cluster instead. GPU clustering has dropped in complexity a lot over the years. Thanks to tools like Ray and libraries like DeepSpeed, you can distribute workloads across multiple smaller GPUs.

Example:

• 3x RTX 4060 Ti at $400 each = $1,200
• 24GB total VRAM, less performance than a 4090, but you now have a parallel setup.

This approach isn’t for everyone (you’ll need to mess with configs and interconnect bandwidth), but modern frameworks like PEFT+LoRA are cluster-friendly out of the box.

Not into physical setups? Just buy some cloud compute time. Hetzner gives you RTX 3090 cloud instances around $1.70/hour (check availability first, they sell out fast). DO, Linode, or even vast.ai are options too.

3. Used Pro Cards for the Win #

Wait for it: used data center GPUs like A100s and A6000s are all over eBay. The A100 with 40GB VRAM regularly drops into the sub-$4K range, and it’s a beast for ML workloads. Meanwhile, 3090s with dual-slot coolers are just screaming deals these days — I saw one for $800 last week.

No RGB, no DLSS for your gaming needs, sure. But as a raw compute workhorse? Hard to beat. They just work, and for professional use, this is where the smarter money lands.

When NOT to Listen to Me #

• You’re rich, and $9,000 doesn’t matter. Cool, buy the RTX 5090 and build the death machine of your dreams.
• You’re bottlenecked specifically on VRAM. Running non-quantized 65B models locally or full LoRA on high parameter outputs? Yeah, big GPUs might be worth it.
• Electricity is dirt cheap for you. 3x 4060 Tis in a cluster will suck more power than one RTX 5090. Something to think about.

But for 95% of us? Nah, skip the 5090 flex.

Quick FAQs #

Can I run modern LLMs on a 4090? #

Yes. Most 30B models (LLaMA 2, Mistral, etc.) run perfectly fine on 24GB VRAM with modern quantization techniques like GPTQ or bitsandbytes. Just don’t expect miracles beyond 65B models.

What about AMD cards? #

AMD’s ROCm ecosystem is still too immature for local LLM community workflows. It works for large batch inferencing in specific cases, but a lot of LLM tooling assumes CUDA.

What’s the best budget GPU for AI tasks? #

The RTX 3060 (12GB VRAM). At ~$300, it’s the go-to pick for running smaller models like 13B quantized or experimenting with fine-tuning on a shoestring budget.

That’s the gist. Don’t blow $9K trying to impress Reddit. Optimize smarter, and laugh all the way to the server bank. Or just put the money into pizza and a 2TB SSD.