llama.cpp

What people said

• “The docs are a great start.”

• “Thanks for all of your exploration in public Simon. Commenting because the fix I proposed was merged in roughly 49 commits after the PrismML Fork. The “tensor API is not supported” warning occurred because llama.cpp’s startup probe fails to compile a matmul2d kernel: Metal’s tens…”

• “With MoE models like gpt-oss, you can run some layers on the CPU (and some on GPU): Mentions 120b is runnable on 8GB VRAM too: "Note that even with just 8GB of VRAM, we can adjust the CPU layers so that we can run the large…”

• “Vision Now Available in Llama.cpp (550 points)”

• “Llama.cpp Now Supports Qwen2-VL (Vision Language Model) (155 points)”

• “Performance of llama.cpp on Apple Silicon A-series (100 points)”

• “LLama.cpp now has a web interface (328 points)”

• “Llama.cpp: Full CUDA GPU Acceleration (728 points)”