—
Mistral AI3B paramsBF16Dense

Ministral-3 3B

Measured on 2026-08-09 with vLLM v0.26.0. In the chat test with ten concurrent requests this profile reaches 42.31 tokens/s per user, with an average of 0.86 seconds to the first token. At 25k context that average wait rises to 15.14 seconds. These runs measure speed and waiting, not the quality of the answers.

—
Suitability score (indicative)
—
Throughput tok/s
8 GB
VRAM
0/11
Reliable benches
Hugging Face →·vLLM v0.26.0·DGX Spark, NVIDIA GB10, 128 GB unified memory·Measured with context 128K·Last measured 9 August 2026
02Vendor quality · reference only

Knowledge: MMLU-Pro; reasoning: GPQA-Diamond; coding: LiveCodeBench v6. Each value names its exact test. Other tests and missing measurements do not qualify for the total score. External figures are not our own evaluation of this precision. Source: vendor model card ↗

3/3
Coverage
70.7
MMLU
53.4
GPQA-Diamond
54.8
LiveCodeBench
03Performance · BF16

Decode throughput · total t/s · c=10

■ BF16
1k ctx BF16—
8k ctx BF16—
4k+turn BF16—
25k ctx BF16—
04Test suite · 11 benchmarks

6 closed-loop tests with llama-benchy and 5 open-loop tests with vllm bench serve. Only complete runs without failed sanity checks are eligible; open-loop tests require at least 99% successful requests. Raw results remain visible. Expand “view command” for the reconstructed command. Methodology →

01 · llama-benchyclosed-loop

Chat

Short prompt, long answer. The shape that has to feel like ordinary chat; TTFT decides whether it feels snappy.

pp (prompt)
1024
tg (gen)
1024
depth
0
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
42.3t/s
TTFT · mean
864ms
Sanity check failed
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model ministral-3-3b-instruct-bf16 \
  --pp 1024 \
  --tg 1024 \
  --depth 0 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
02 · llama-benchyclosed-loop

RAG · 8k context

Medium context, a few document chunks with an answer of normal length. Shows prefill cost without hitting the wall.

pp (prompt)
8192
tg (gen)
512
depth
0
concurrency
5 · 10 · 20
repeats
3
Tokens/sec · per user
17.5t/s
TTFT · mean
3.4s
Sanity check failed
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model ministral-3-3b-instruct-bf16 \
  --pp 8192 \
  --tg 512 \
  --depth 0 \
  --concurrency 5 10 20 \
  --runs 3 \
  --latency-mode generation \
  --format md
03 · llama-benchyclosed-loop

Long output and agents

Short instruction, lots of output. Code generation, reports or structured agent output. A stress test for decode throughput.

pp (prompt)
256
tg (gen)
4096
depth
0
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
50.2t/s
TTFT · mean
461ms
Sanity check failed
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model ministral-3-3b-instruct-bf16 \
  --pp 256 \
  --tg 4096 \
  --depth 0 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
04 · llama-benchyclosed-loop

Multi-turn office work

Five turns per conversation, ten conversations in parallel. Close to how a team actually uses this, with context growing every turn.

pp (prompt)
2048
tg (gen)
512
depth
4
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
35t/s
TTFT · mean
1.21s
Sanity check failed
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model ministral-3-3b-instruct-bf16 \
  --pp 2048 \
  --tg 512 \
  --depth 4 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
05 · llama-benchyclosed-loop

Large context · 25k

A stress test with large prompts. Not necessarily chat material, but exactly where the prefill wall shows up and TTFT collapses.

pp (prompt)
4096 · 8192 · 16384 · 25000
tg (gen)
256
depth
0
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
6.3t/s
TTFT · mean
15.14s
Sanity check failed
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model ministral-3-3b-instruct-bf16 \
  --pp 4096 8192 16384 25000 \
  --tg 256 \
  --depth 0 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
06 · llama-benchyclosed-loop

Concurrency under pressure

25k context with twenty requests at once. Five and ten are already covered by test 05; twenty shows where the scheduler gives up.

pp (prompt)
25000
tg (gen)
256
depth
0
concurrency
20
repeats
3
Tokens/sec · per user
3.2t/s
TTFT · mean
31.19s
Sanity check failed
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model ministral-3-3b-instruct-bf16 \
  --pp 25000 \
  --tg 256 \
  --depth 0 \
  --concurrency 20 \
  --runs 3 \
  --latency-mode generation \
  --format md
07 · vllm bench serveopen-loop

Realistic office baseline

Random dataset, 4000 tokens in and 500 out, arrival rate 0.3 with burstiness 0.7. A quiet office.

dataset
random
input / output
4000 / 500
rate (req/s)
0.3
prompts
200
burstiness
0.7
Tokens/sec · per user
95.1t/s
TTFT · p50
660ms
Request success 100% · Sanity check failed
200 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 200 \
  --request-rate 0.3 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 07-office-baseline.json
08 · vllm bench serveopen-loop

Real conversations · ShareGPT

ShareGPT V3, averaging 228 tokens per turn, varying naturally per conversation. What real users do, not a synthetic distribution.

dataset
sharegpt
rate (req/s)
0.3
prompts
250
burstiness
0.7
Tokens/sec · per user
18.1t/s
TTFT · p50
69ms
Request success 100% · Sanity check failed
250 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name sharegpt \
  --dataset-path /tmp/ShareGPT_V3.json \
  --num-prompts 250 \
  --request-rate 0.3 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 08-sharegpt.json
09 · vllm bench serveopen-loop

Reasoning workload

Long chains of thought, 1k in and 4k out, a slow arrival rate of 0.2 because every request costs a lot of decode budget. Tests whether TTFT stays stable.

dataset
random
input / output
1024 / 4096
rate (req/s)
0.2
prompts
50
burstiness
1
Tokens/sec · per user
11.4t/s
TTFT · p50
423ms
Request success 100% · Sanity check failed
50 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 1024 \
  --random-output-len 4096 \
  --random-range-ratio 0.9 \
  --num-prompts 50 \
  --request-rate 0.2 \
  --burstiness 1.0 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 09-reasoning.json
10 · vllm bench serveopen-loop

Monday morning peak

Random, 4000 in and 500 out, arrival rate 1.5 per second with burstiness 1.0 and at most 25 in parallel. A heavy, independent request stream.

dataset
random
input / output
4000 / 500
rate (req/s)
1.5
prompts
300
burstiness
1
max parallel
25
Tokens/sec · per user
80.5t/s
TTFT · p50
803ms
Request success 100% · Sanity check failed
300 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 300 \
  --request-rate 1.5 \
  --burstiness 1.0 \
  --max-concurrency 25 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 10-monday-peak.json
11 · vllm bench serveopen-loop

Capacity · rate sweep

Six increasing arrival rates, from 0.1 to 1.0 requests per second. At each rung we check whether p95 TTFT still stays under the threshold.

dataset
random
input / output
4000 / 500
rates (req/s)
0.1 · 0.2 · 0.3 · 0.5 · 0.7 · 1.0
prompts
100 · 100 · 100 · 125 · 175 · 250
burstiness
0.7
Req/s · TTFT < 5s
0.46req/s
Req/s · TTFT < 10s
0.46req/s
Request success 100% · Sanity check failed

Stopped at 1.0 req/s. Capacity is a lower bound, not a measured ceiling.

6 rates · seed 42view command →hide command ↑ 
# 0.1 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 100 \
  --request-rate 0.1 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.1.json

# 0.2 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 100 \
  --request-rate 0.2 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.2.json

# 0.3 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 100 \
  --request-rate 0.3 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.3.json

# 0.5 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 125 \
  --request-rate 0.5 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.5.json

# 0.7 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 175 \
  --request-rate 0.7 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.7.json

# 1.0 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model mistralai/Ministral-3-3B-Instruct-2512 \
  --tokenizer mistralai/Ministral-3-3B-Instruct-2512 \
  --served-model-name ministral-3-3b-instruct-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 250 \
  --request-rate 1.0 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-1.0.json
05Notes on this run

Chat

Chat with ten concurrent requests

The chat measurement is excluded because the sanity check failed. The tokens/s present therefore prove no usable model output.

Long context

Wait at 25k context

The test with 25k input tokens is excluded as well. The raw results sit with the test, but they are no valid basis for a performance recommendation.

Peak load

Requests and wait under peak load

The peak measurement is excluded by the failed sanity check. A successful HTTP request is not the same as a usable answer.

Measurement coverage

Which measurements count

0 of 11 tests have a measurement that counts under the current Arena rules. The sanity check failed. This run is therefore excluded from the ranking; the raw figures stay visible for inspection only. Aborted: 11-rate-sweep.

More numbers?
Read the full article.

RunSource data and metadata→WeightsHugging Face→BackTo the arena→
MarkdownSource JSON
Explanation