Nemotron 120B (MoE) on a DGX Spark
Running NVIDIA Nemotron-3 Super 120B (A12B, NVFP4) — a Mixture-of-Experts model with ~12B active parameters per token — on a single DGX Spark (GB10). Snapshot from a healthy serving window:
| Metric | Value |
|---|---|
| Status | ✅ up, no errors |
| Generation throughput | ~13–16 tok/s steady |
| Prompt throughput | bursts up to 2,400 tok/s (prefix-cached) |
| KV cache usage | 7.9–9.6% |
| Prefix cache hit rate | 62–63% |
| Active requests | 0–1, all 200 OK |
Bottom line: comfortable. KV under 10% means plenty of headroom for longer prompts, and ~15 tok/s is right where you’d expect a 12B-active MoE to land on this box. This is my heavy long-context reasoning engine.