Qwen2.5-3B on Ollama: keep it simple

A small, fast model — Qwen2.5-3B (65k context) — served through Ollama, benchmarked to decide whether it was worth moving to llama.cpp.

MetricMedian
Output throughput71 tok/s
TTFT1.75 s
Request latency3.51 s
Failures0

Results were excellent and extremely consistent across runs.

Bottom line: keep Ollama. There’s no performance reason to move to llama.cpp for this workload — spend the effort testing alternative models for quality instead, not swapping the runtime. The laziest stack that hits the numbers wins.