Qwen2.5-3B on Ollama: keep it simple
A small, fast model — Qwen2.5-3B (65k context) — served through Ollama, benchmarked to decide whether it was worth moving to llama.cpp.
| Metric | Median |
|---|---|
| Output throughput | 71 tok/s |
| TTFT | 1.75 s |
| Request latency | 3.51 s |
| Failures | 0 |
Results were excellent and extremely consistent across runs.
Bottom line: keep Ollama. There’s no performance reason to move to llama.cpp for this workload — spend the effort testing alternative models for quality instead, not swapping the runtime. The laziest stack that hits the numbers wins.