37 tok/s from Qwen3-Next-80B (48 GB) on a 32 GB laptop, 24 without drafts vs 2.8 for llama.cpp; reading 62 vs 16 tok/s. Faster than llama.cpp on all 13 models compared in Q4_K_M. LLMs bigger than your RAM, pure C, CPU or GPU.
- Recommended by
- 2
- GitHub stars
- 572
- Licence
- GPL-3.0
- Language
- C
- Last commit
- 30 Oct 2026
What people said
“Then check out those: <- this, focuses on Qwen3-Next-80B-A3B. <- Qwen3.8-Flash-Next. I use this (tuned) and I cannot understate how satisfied I am. <- DeepSeek 4 family + more.…”
“Janas: Run LLMs larger than RAM by streaming experts from SSD (221 points)”
Not software, abandoned or listed by mistake? Report this listing