{"name":"Nemotron-3-Super-120B-A12B","aaUrl":"https://artificialanalysis.ai/models/nvidia-nemotron-3-super-120b-a12b","arena":{"id":"nemotron-3-super-nvfp4","name":"Nemotron-3-Super 120B-A12B","vram":60,"params":120,"vendor":"nvidia","quality":{"coding":{"bench":"LiveCodeBench v5","value":81.2},"science":{"bench":"GPQA","value":79.2},"knowledge":{"bench":"MMLU-Pro","value":83.7}},"contextK":256},"notes":"Model-ID: nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4. Container: vllm/vllm-openai:v0.26.0-aarch64-cu129-ubuntu2404. llama-benchy 0.4.0. Bronrun: 2026-08-15T04:22:40+02:00. KV-cache fp8; max_model_len 131072; gpu_memory_utilization 0,9. Prefix caching uit; async scheduling aan. Extra serverflags: --trust-remote-code --enable-chunked-prefill --max-num-seqs 20 --moe-backend marlin --mamba_ssm_cache_dtype float32 --quantization fp4 --speculative_config {\"method\":\"mtp\",\"num_speculative_tokens\":3,\"moe_backend\":\"triton\"} --reasoning-parser-plugin /app/super_v3_reasoning_parser.py --reasoning-parser super_v3 --enable-auto-tool-choice --tool-call-parser qwen3_coder --linear-backend marlin. Profielvariabelen:  VLLM_TEST_FORCE_FP8_MARLIN=1 VLLM_ALLOW_LONG_MAX_MODEL_LEN=1 VLLM_FLASHINFER_ALLREDUCE_BACKEND=trtllm VLLM_USE_FLASHINFER_MOE_FP4=0. 11 van de 11 tests hebben een meting die volgens de huidige Arena-regels meetelt. De sanity-check is geslaagd. Dit is een controle op een kort gegenereerd antwoord, geen inhoudelijke evaluatie van de benchmarktaken.","order":9,"engine":"vLLM v0.26.0","verdict":"Gemeten op 2026-08-15 met vLLM v0.26.0. In de chattest met tien gelijktijdige verzoeken haalt dit profiel 8,1 tokens/s per gebruiker, met gemiddeld 6,12 seconden tot het eerste token. Bij 25k context stijgt die gemiddelde wachttijd naar 103,43 seconden. Deze runs meten snelheid en wachttijd, niet de kwaliteit van de antwoorden.","findings":{"broke":{"body":"Bij 25000 inputtokens en 256 outputtokens met tien gelijktijdige verzoeken is de gemiddelde wachttijd voor het eerste token 103,43 seconden. Daarna genereert het model gemiddeld 3,83 tokens/s per gebruiker. De meting alleen toont niet of geheugen, promptverwerking of scheduling de oorzaak van vertraging is.","title":"Wachttijd bij 25k context"},"costly":{"body":"De run verwerkt 0,14 verzoeken/s bij een ingestelde aankomstsnelheid van 1,5 verzoeken/s. 300 van de 300 verzoeken slagen. De p95-wachttijd voor het eerste token is 56,83 seconden.","title":"Verzoeken en wachttijd bij piekbelasting"},"worked":{"body":"Bij 1024 input- en 1024 outputtokens met tien gelijktijdige verzoeken meet ik gemiddeld 8,1 tokens/s per gebruiker. De gemiddelde wachttijd voor het eerste token is 6,12 seconden.","title":"Chat bij tien gelijktijdige verzoeken"},"surprised":{"body":"11 van de 11 tests hebben een meting die volgens de huidige Arena-regels meetelt. De sanity-check is geslaagd. Dit is een controle op een kort gegenereerd antwoord, geen inhoudelijke evaluatie van de benchmarktaken.","title":"Welke metingen meetellen"}},"hardware":"DGX Spark, NVIDIA GB10, 128 GB unified memory","modelUrl":"https://huggingface.co/nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4","provider":"NVIDIA","precision":"NVFP4","parameters":"120B (12B active)","provenance":{"modelId":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4","runPath":"results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4","validity":{"tests":{"01-chat":"complete","02-rag-8k":"complete","08-sharegpt":"complete","09-reasoning":"complete","04-multi-turn":"complete","11-rate-sweep":"complete","03-long-output":"complete","05-big-context":"complete","10-monday-peak":"complete","07-office-baseline":"complete","06-concurrency-stress":"complete"},"sanity":"passed"},"sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","servedName":"nemotron-3-super","generatedAt":"2026-08-15T04:22:40+02:00","runRevision":"a948cec23b054c62fef595bad3cdb284066922ac","vllmVersion":"0.26.0","suiteVersion":"2026-08","repositoryUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks","contractVersion":1,"contractRevision":"a948cec23b054c62fef595bad3cdb284066922ac","commandProvenance":"reconstructed_from_raw_artifacts","llamaBenchyVersion":"0.4.0"},"architecture":"MoE","measuredContextK":128,"benchmarkResults":[{"id":"nemotron-3-super-120b-a12b-nvfp4-chat-1k","date":"2026-08-15","runs":3,"metrics":{"ttfrSec":{"mean":6.115,"stddev":1.892},"decodeTokensPerSecondTotal":{"mean":72.58,"stddev":1.1},"prefillTokensPerSecondTotal":{"mean":1129.65,"stddev":1.4},"decodeTokensPerSecondPerUser":{"mean":8.1,"stddev":0.39}},"sourceRun":"Test 01","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","workloadId":"chat-1k","sourceLabel":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 nvfp4 op de DGX Spark","sourceStatus":"complete","sourceTestId":"01-chat","prefixCaching":false},{"id":"nemotron-3-super-120b-a12b-nvfp4-rag-8k","date":"2026-08-15","runs":3,"metrics":{"ttfrSec":{"mean":34.652,"stddev":16.838},"decodeTokensPerSecondTotal":{"mean":43.95,"stddev":0.79},"prefillTokensPerSecondTotal":{"mean":1225.89,"stddev":0.75},"decodeTokensPerSecondPerUser":{"mean":6.35,"stddev":1.15}},"sourceRun":"Test 02","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","workloadId":"rag-8k","sourceLabel":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 nvfp4 op de DGX Spark","sourceStatus":"complete","sourceTestId":"02-rag-8k","prefixCaching":false},{"id":"nemotron-3-super-120b-a12b-nvfp4-long-output-agents","date":"2026-08-15","runs":3,"metrics":{"ttfrSec":{"mean":2.396,"stddev":0.143},"decodeTokensPerSecondTotal":{"mean":60.1,"stddev":9.79},"prefillTokensPerSecondTotal":{"mean":878.92,"stddev":21.71},"decodeTokensPerSecondPerUser":{"mean":8.62,"stddev":1.5}},"sourceRun":"Test 03","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","workloadId":"long-output-agents","sourceLabel":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 nvfp4 op de DGX Spark","sourceStatus":"complete","sourceTestId":"03-long-output","prefixCaching":false},{"id":"nemotron-3-super-120b-a12b-nvfp4-multi-turn-office","date":"2026-08-15","runs":3,"metrics":{"ttfrSec":{"mean":10.061,"stddev":4.254},"decodeTokensPerSecondTotal":{"mean":63.66,"stddev":0.4},"prefillTokensPerSecondTotal":{"mean":1196.77,"stddev":5.19},"decodeTokensPerSecondPerUser":{"mean":7.6,"stddev":0.55}},"sourceRun":"Test 04","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","workloadId":"multi-turn-office","sourceLabel":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 nvfp4 op de DGX Spark","sourceStatus":"complete","sourceTestId":"04-multi-turn","prefixCaching":false},{"id":"nemotron-3-super-120b-a12b-nvfp4-large-context-25k","date":"2026-08-15","runs":3,"metrics":{"ttfrSec":{"mean":103.429,"stddev":52.998},"decodeTokensPerSecondTotal":{"mean":13.35,"stddev":0.35},"prefillTokensPerSecondTotal":{"mean":1211.74,"stddev":0.38},"decodeTokensPerSecondPerUser":{"mean":3.83,"stddev":2.98}},"sourceRun":"Test 05","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","workloadId":"large-context-25k","sourceLabel":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 nvfp4 op de DGX Spark","sourceStatus":"complete","sourceTestId":"05-big-context","prefixCaching":false},{"id":"nemotron-3-super-120b-a12b-nvfp4-concurrency-stress-25k","date":"2026-08-15","runs":3,"metrics":{"ttfrSec":{"mean":198.114,"stddev":109.005},"decodeTokensPerSecondTotal":{"mean":13.04,"stddev":0.15},"prefillTokensPerSecondTotal":{"mean":1192.35,"stddev":2.15},"decodeTokensPerSecondPerUser":{"mean":2.59,"stddev":2.41}},"sourceRun":"Test 06","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","workloadId":"concurrency-stress-25k","sourceLabel":"nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-NVFP4 nvfp4 op de DGX Spark","sourceStatus":"complete","sourceTestId":"06-concurrency-stress","prefixCaching":false}],"openLoopResults":[{"id":"nemotron-3-super-120b-a12b-nvfp4-office-random-4k","date":"2026-08-15","metrics":{"e2eMs":{"p50":453335.79,"p90":734352.85,"p95":765615.19,"p99":796107.91,"mean":456244.83},"tpotMs":{"p50":274.51,"p90":314.2,"p95":325.56,"p99":346.22,"mean":269.83},"ttftMs":{"p50":339386.6,"p90":607244.62,"p95":640622.64,"p99":672869.32,"mean":326344.45}},"sourceRun":"Test 07","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","scenarioId":"office-random-4k","achievedRps":0.14,"sourceStatus":"complete","sourceTestId":"07-office-baseline","configuredRps":0.3,"prefixCaching":false,"totalRequests":200,"successfulRequests":200,"totalTokenThroughput":638.75,"peakConcurrentRequests":119},{"id":"nemotron-3-super-120b-a12b-nvfp4-sharegpt-replay","date":"2026-08-15","metrics":{"e2eMs":{"p50":17722.2,"p90":74848.65,"p95":88796.64,"p99":116544.6,"mean":29339.91},"tpotMs":{"p50":135.14,"p90":165.96,"p95":180.11,"p99":200.31,"mean":130.39},"ttftMs":{"p50":1089.43,"p90":1521.62,"p95":1790.27,"p99":1990.92,"mean":1135.29}},"sourceRun":"Test 08","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","scenarioId":"sharegpt-replay","achievedRps":0.294,"sourceStatus":"complete","sourceTestId":"08-sharegpt","configuredRps":0.3,"prefixCaching":false,"totalRequests":250,"successfulRequests":250,"totalTokenThroughput":129.07,"peakConcurrentRequests":18},{"id":"nemotron-3-super-120b-a12b-nvfp4-reasoning","date":"2026-08-15","metrics":{"e2eMs":{"p50":788425.38,"p90":1139031.87,"p95":1189419.14,"p99":1212066.83,"mean":743187.44},"tpotMs":{"p50":118.35,"p90":138.63,"p95":157.59,"p99":168.45,"mean":119.58},"ttftMs":{"p50":284459.11,"p90":653846.58,"p95":673895.06,"p99":737928.34,"mean":266828.83}},"sourceRun":"Test 09","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","scenarioId":"reasoning","achievedRps":0.035,"sourceStatus":"complete","sourceTestId":"09-reasoning","configuredRps":0.2,"prefixCaching":false,"totalRequests":50,"successfulRequests":50,"totalTokenThroughput":184,"peakConcurrentRequests":48},{"id":"nemotron-3-super-120b-a12b-nvfp4-monday-peak-random-4k","date":"2026-08-15","metrics":{"e2eMs":{"p50":171906.22,"p90":268743.75,"p95":286221.11,"p99":315207.49,"mean":172341.33},"tpotMs":{"p50":272.57,"p90":322.19,"p95":346.69,"p99":408.23,"mean":272.84},"ttftMs":{"p50":38453.01,"p90":52512.56,"p95":56834.12,"p99":84930.34,"mean":40014.02}},"sourceRun":"Test 10","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","scenarioId":"monday-peak-random-4k","achievedRps":0.141,"sourceStatus":"complete","sourceTestId":"10-monday-peak","configuredRps":1.5,"prefixCaching":false,"totalRequests":300,"successfulRequests":300,"totalTokenThroughput":636.57,"peakConcurrentRequests":27}],"rateSweep":{"date":"2026-08-15","capacity":{"slo2000":null,"slo5000":null,"slo10000":null},"sourceRun":"Test 11","sourceUrl":"https://github.com/djangodevreng/dgx-spark-benchmarks/tree/a948cec23b054c62fef595bad3cdb284066922ac/results/nemotron-3/nemotron-3-super-120b-a12b/nvfp4/","abortedAtRps":null,"queueKneeRps":null,"sourceStatus":"complete","sourceTestId":"11-rate-sweep","totalRequests":850,"successfulRequests":850}}