Can you actually buy LLMs at a regular supermarket?

While you cannot buy large language models in the grocery aisle, some electronics megastores and online retailers have started selling SD cards pre-loaded with local AI models designed to run on single-board computers like the Raspberry Pi 5.

Is an SD card fast enough to run local LLMs?

No. Cheap Class 10 SD cards have terrible IOPS, which drastically limits prompt processing speeds and can bottleneck your tokens per second to unusable levels. You need fast NVMe storage for a smooth local LLM experience.

Should I build a local AI PC or rent cloud compute?

If you only run small models (under 14B parameters), a local setup like a Mac Mini or Raspberry Pi with NVMe storage is great. For larger models (70B+), renting hourly compute on platforms like Hetzner or Vast.ai is significantly cheaper than buying expensive dedicated GPUs.

Buying LLMs at the Supermarket: The Hardware Reality of Toy Tapes

Table of Contents

There was a post on r/LocalLLaMA recently that stopped my scroll. User u/smol_ai_tinkerer found SD cards labeled “AI Brains” hanging on a pegboard in a European electronics megastore, right next to the’)==…

Okay, let’s back up. We aren’t quite at LLMs in the produce aisle yet, but the premise of the thread hit a nerve. Someone found pre-loaded SD cards packed with quantized models for the Raspberry Pi 5, marketed to完全 normies.

The thread blew up because it touches the raw nerve of our community: the hardware barrier to entry. The debate split into two camps. The purists want to build. The pragmatists just want it to work. Let’s break down the actual approaches.

The Blueprint Approach: EMMC vs SD Card Speeds #

If you buy a pre-loaded card, you are frankly cooking with microwave fire.

The top comment in the thread pointed out the glaring flaw: running an 8-bit quantized 3B model from a cheap Class 10 SD card gives you atrocious prompt processing speeds. We are talking 2 tokens per second. Painful. You need decent IOPS.

If you are dead set on a SBC setup, skip the pre-packaged SD card. Buy a Pinebook Pro or a Pi 5 with an NVMe baseboard. It is overkill for casual users, but mandatory if you actually want to use the model without falling asleep. You need fast storage. I haven’t tested this on the new Rock 5B yet, but my Pi 5 with an NVMe drive hits 12 t/s on Llama 3 8B Q4_K_M. That’s the bare minimum for a usable chat.

The Managed Route: Skip the NUC, Rent the Compute #

Then we have the cowards. Just kidding—this is actually the smart play right now.

User u/quantized_ron made an excellent point in the replies: by the time you buy a 16GB Mac Mini to run a local 8B model, you could have rented a cloud endpoint for two years. I love self-hosting, but for any model over 14B parameters, local hardware gets stupid expensive.

I run my own local models on a single, aging M1 Pro using Ollama. It handles Q4 8B fine. For anything bigger? I just spin up a Hetzner box. You can get a dedicated machine with an RTX 4000 for under fifty cents an hour. DigitalOcean is a complete rip-off for this use case. Do not touch them. Use Hetzner. Use Vast.ai if you are okay with potential reliability issues.

The community is genuinely split on this. Half of r/LocalLLaMA thinks if the silicon isn’t inside your house, it doesn’t count. The other half just wants to do batch inference on a 70B fine-tune without losing RAM. Be honest with what you want. If you want a toy, buy the supermarket SD card. If you want a tool, rent compute until Apple silicon or NVIDIA decides to stop robbing us blind.