https://www.smart-stacking.com/tags/llm/Recent content in Llm on SmartStack: AI, Self-Hosting & Smart FinanceHugoen© 2026 SmartStackThu, 06 Aug 2026 02:00:31 +0800https://www.smart-stacking.com/posts/2026-08-06-minimax-issues/Thu, 06 Aug 2026 02:00:31 +0800https://www.smart-stacking.com/posts/2026-08-06-minimax-issues/MiniMax is boasting a 1M token context, but r/LocalLLaMA found the API math and endpoint quirks unmatched.https://www.smart-stacking.com/posts/2026-08-05-kimi-k3-full-model-running-on-16x-gb10-cluster-at-20tps/Wed, 05 Aug 2026 14:00:30 +0800https://www.smart-stacking.com/posts/2026-08-05-kimi-k3-full-model-running-on-16x-gb10-cluster-at-20tps/Kimi K3 hums at 20+ TPS on a 16x GB10 cluster. I tried it. Here is why it is brilliant and entirely impractical.https://www.smart-stacking.com/posts/2026-08-04-only-3-days-ago/Tue, 04 Aug 2026 08:57:22 +0800https://www.smart-stacking.com/posts/2026-08-04-only-3-days-ago/A deep dive into spinning up Qwen2.5-72B on Hetzner using vLLM on a single 80GB GPU. Numbers, configs, and why it might be totally overkill for you.https://www.smart-stacking.com/posts/2026-08-04-daniel-han-of-unsloth-validates-qwen38-27b-will-run-only-17gb-vram/Tue, 04 Aug 2026 00:49:11 +0800https://www.smart-stacking.com/posts/2026-08-04-daniel-han-of-unsloth-validates-qwen38-27b-will-run-only-17gb-vram/Daniel Han claims a 27B model runs on 17GB VRAM. We look at the benchmarks, the compromises, and if your 3090 can survive it.https://www.smart-stacking.com/posts/2026-08-03-deepseek-v4-flash-0731-surpasses-fable-5-sol-kimi-k3-on-chess-benchmark/Mon, 03 Aug 2026 04:32:07 +0800https://www.smart-stacking.com/posts/2026-08-03-deepseek-v4-flash-0731-surpasses-fable-5-sol-kimi-k3-on-chess-benchmark/DeepSeek-V4-Flash-0731 just crushed Fable-5 and Kimi-K3 on the Chess Benchmark. Here is why that does not matter.https://www.smart-stacking.com/posts/2026-08-03-llamacpp-just-added-mtp-dspark-support-for-deepseek-v4-flash/Mon, 03 Aug 2026 02:30:06 +0800https://www.smart-stacking.com/posts/2026-08-03-llamacpp-just-added-mtp-dspark-support-for-deepseek-v4-flash/llama.cpp just landed MTP / DSpark support for DeepSeek V4 Flash. I spent the weekend breaking my rig to test it.https://www.smart-stacking.com/posts/2026-08-02-i-pushed-kimi-k3-onto-one-cpu-with-8-gb-of-ram/Sun, 02 Aug 2026 14:19:04 +0800https://www.smart-stacking.com/posts/2026-08-02-i-pushed-kimi-k3-onto-one-cpu-with-8-gb-of-ram/I tried running Kimi K3 on a single 8GB CPU. Here is exactly how badly it went and what you should run instead.https://www.smart-stacking.com/posts/2026-08-02-ran-ds-v4-flash-0731-locally-on-3xmi50-32gb-15-ts-tg/Sun, 02 Aug 2026 12:18:03 +0800https://www.smart-stacking.com/posts/2026-08-02-ran-ds-v4-flash-0731-locally-on-3xmi50-32gb-15-ts-tg/Running DS V4-Flash-0731 on cheap eBay MI50s actually hits 15 t/s. Here is how the r/LocalLLaMA pojects make this work.https://www.smart-stacking.com/posts/2026-08-02-deepseek-v4-flash-0731-ud-iq3s-125-toks-on-rtx-3090-128gb-ddr5/Sun, 02 Aug 2026 10:15:03 +0800https://www.smart-stacking.com/posts/2026-08-02-deepseek-v4-flash-0731-ud-iq3s-125-toks-on-rtx-3090-128gb-ddr5/A no-BS guide to squeezing 12.5 tok/s out of DeepSeek-V4-Flash on a 3090 and 128GB DDR5 using IQ3_S quantization.https://www.smart-stacking.com/posts/2026-08-02-eu-ai-act-takes-effect-tomorrow-august-2-2026/Sun, 02 Aug 2026 06:13:02 +0800https://www.smart-stacking.com/posts/2026-08-02-eu-ai-act-takes-effect-tomorrow-august-2-2026/EU AI Act officially hits August 2, 2026. Here is what it actually means for local LLM nerds running models on homelab GPUs.https://www.smart-stacking.com/posts/2026-08-01-deepseek-v4-flash-0731-models-you-can-run-locally-now-have-the-intelligence-score-of-the-top-frontier-model-from-march-2026/Sat, 01 Aug 2026 22:07:00 +0800https://www.smart-stacking.com/posts/2026-08-01-deepseek-v4-flash-0731-models-you-can-run-locally-now-have-the-intelligence-score-of-the-top-frontier-model-from-march-2026/DeepSeek-V4-Flash-0731 brings March 2026 frontier intelligence to local GPUs. Here is how to actually run it without burning your house down.https://www.smart-stacking.com/posts/2026-08-01-me-worn-out-from-all-the-new-model-drops-this-week-but-still-hyped-for-all-the-great-new-releases/Sat, 01 Aug 2026 13:58:59 +0800https://www.smart-stacking.com/posts/2026-08-01-me-worn-out-from-all-the-new-model-drops-this-week-but-still-hyped-for-all-the-great-new-releases/Worn out from this week’s model drops? Here is how to actually run the new 70Bs without burning out your RAM or your wallet.https://www.smart-stacking.com/posts/2026-08-01-deepseek-v4-flash-ga-ranks-the-same-as-sonnet-5-and-grok-45-on-deepswe/Sat, 01 Aug 2026 09:55:58 +0800https://www.smart-stacking.com/posts/2026-08-01-deepseek-v4-flash-ga-ranks-the-same-as-sonnet-5-and-grok-45-on-deepswe/DeepSeek V4 Flash GA just tied Sonnet 5 and Grok 4.5 on DeepSWE. The community is skeptical, and the VRAM math is brutal.https://www.smart-stacking.com/posts/2026-07-31-the-chinese-llm-release-carousel-never-stops-place-your-bets-for-minimax-next-week/Fri, 31 Jul 2026 23:47:55 +0800https://www.smart-stacking.com/posts/2026-07-31-the-chinese-llm-release-carousel-never-stops-place-your-bets-for-minimax-next-week/Qwen and DeepSeek broke the bank, now MiniMax is stepping up to the plate next week. Here is why you should care.https://www.smart-stacking.com/posts/2026-07-31-deepseek-aideepseek-v4-flash-0731-on-huggingface/Fri, 31 Jul 2026 21:45:55 +0800https://www.smart-stacking.com/posts/2026-07-31-deepseek-aideepseek-v4-flash-0731-on-huggingface/DeepSeek drops a mid-summer Flash update that destroys Llama 4 in tokens/sec, but your GPU might actually catch fire.https://www.smart-stacking.com/posts/2026-07-31-deepseek-v4-flash-has-been-updated-the-official-release-of-deepseek-v4-pro-will-follow-soon/Fri, 31 Jul 2026 17:41:55 +0800https://www.smart-stacking.com/posts/2026-07-31-deepseek-v4-flash-has-been-updated-the-official-release-of-deepseek-v4-pro-will-follow-soon/DeepSeek-V4-Flash got a silent update and Pro is coming. Here’s what the r/LocalLLaMA trenches actually think about VRAM and speed.https://www.smart-stacking.com/posts/2026-07-31-anthropic-our-models-hacked-three-different-external-companies-months-before-openais-model-was-able-to-do-the-same/Fri, 31 Jul 2026 11:34:52 +0800https://www.smart-stacking.com/posts/2026-07-31-anthropic-our-models-hacked-three-different-external-companies-months-before-openais-model-was-able-to-do-the-same/Anthropic revealed Claude successfully breached three external companies months before OpenAI’s o1 pulled the same stunt. Here is why nobody was ready.