16
google4.5B effectief paramsBF16Dense

Gemma-4 E4B

Measured on 2026-08-17 with vLLM v0.26.0. In the chat test with ten concurrent requests this profile reaches 21.47 tokens/s per user, with an average of 1.05 seconds to the first token. At 25k context that average wait rises to 29.43 seconds. These runs measure speed and waiting, not the quality of the answers.

54.1
Suitability score (indicative)
101
Throughput tok/s
9 GB
VRAM
11/11
Reliable benches
Hugging Face →·vLLM v0.26.0·DGX Spark, NVIDIA GB10, 128 GB unified memory·Measured with context 128K·Last measured 17 August 2026
02Vendor quality · reference only

Knowledge: MMLU-Pro; reasoning: GPQA-Diamond; coding: LiveCodeBench v6. Each value names its exact test. Other tests and missing measurements do not qualify for the total score. External figures are not our own evaluation of this precision. Source: vendor model card ↗

3/3
Coverage
69.4
MMLU-Pro
58.6
GPQA-Diamond
52
LiveCodeBench v6
03Performance · BF16

Decode throughput · total t/s · c=10

■ BF16
1k ctx BF16134 t/s
8k ctx BF16118 t/s
4k+turn BF16145 t/s
25k ctx BF1639 t/s
04Test suite · 11 benchmarks

6 closed-loop tests with llama-benchy and 5 open-loop tests with vllm bench serve. Only complete runs without failed sanity checks are eligible; open-loop tests require at least 99% successful requests. Raw results remain visible. Expand “view command” for the reconstructed command. Methodology →

01 · llama-benchyclosed-loop

Chat

Short prompt, long answer. The shape that has to feel like ordinary chat; TTFT decides whether it feels snappy.

pp (prompt)
1024
tg (gen)
1024
depth
0
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
21.5t/s
TTFT · mean
1.05s
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model gemma-4-e4b-it-bf16 \
  --pp 1024 \
  --tg 1024 \
  --depth 0 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
02 · llama-benchyclosed-loop

RAG · 8k context

Medium context, a few document chunks with an answer of normal length. Shows prefill cost without hitting the wall.

pp (prompt)
8192
tg (gen)
512
depth
0
concurrency
5 · 10 · 20
repeats
3
Tokens/sec · per user
16t/s
TTFT · mean
6.94s
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model gemma-4-e4b-it-bf16 \
  --pp 8192 \
  --tg 512 \
  --depth 0 \
  --concurrency 5 10 20 \
  --runs 3 \
  --latency-mode generation \
  --format md
03 · llama-benchyclosed-loop

Long output and agents

Short instruction, lots of output. Code generation, reports or structured agent output. A stress test for decode throughput.

pp (prompt)
256
tg (gen)
4096
depth
0
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
22.5t/s
TTFT · mean
403ms
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model gemma-4-e4b-it-bf16 \
  --pp 256 \
  --tg 4096 \
  --depth 0 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
04 · llama-benchyclosed-loop

Multi-turn office work

Five turns per conversation, ten conversations in parallel. Close to how a team actually uses this, with context growing every turn.

pp (prompt)
2048
tg (gen)
512
depth
4
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
20.8t/s
TTFT · mean
1.81s
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model gemma-4-e4b-it-bf16 \
  --pp 2048 \
  --tg 512 \
  --depth 4 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
05 · llama-benchyclosed-loop

Large context · 25k

A stress test with large prompts. Not necessarily chat material, but exactly where the prefill wall shows up and TTFT collapses.

pp (prompt)
4096 · 8192 · 16384 · 25000
tg (gen)
256
depth
0
concurrency
1 · 5 · 10
repeats
3
Tokens/sec · per user
7.9t/s
TTFT · mean
29.43s
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model gemma-4-e4b-it-bf16 \
  --pp 4096 8192 16384 25000 \
  --tg 256 \
  --depth 0 \
  --concurrency 1 5 10 \
  --runs 3 \
  --latency-mode generation \
  --format md
06 · llama-benchyclosed-loop

Concurrency under pressure

25k context with twenty requests at once. Five and ten are already covered by test 05; twenty shows where the scheduler gives up.

pp (prompt)
25000
tg (gen)
256
depth
0
concurrency
20
repeats
3
Tokens/sec · per user
4.7t/s
TTFT · mean
55.76s
3 repeats · mean ± stddevview command →hide command ↑ 
uvx llama-benchy==0.4.0 \
  --base-url http://localhost:8000/v1 \
  --model gemma-4-e4b-it-bf16 \
  --pp 25000 \
  --tg 256 \
  --depth 0 \
  --concurrency 20 \
  --runs 3 \
  --latency-mode generation \
  --format md
07 · vllm bench serveopen-loop

Realistic office baseline

Random dataset, 4000 tokens in and 500 out, arrival rate 0.3 with burstiness 0.7. A quiet office.

dataset
random
input / output
4000 / 500
rate (req/s)
0.3
prompts
200
burstiness
0.7
Tokens/sec · per user
81.6t/s
TTFT · p50
889ms
Request success 100% · Included
200 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 200 \
  --request-rate 0.3 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 07-office-baseline.json
08 · vllm bench serveopen-loop

Real conversations · ShareGPT

ShareGPT V3, averaging 228 tokens per turn, varying naturally per conversation. What real users do, not a synthetic distribution.

dataset
sharegpt
rate (req/s)
0.3
prompts
250
burstiness
0.7
Tokens/sec · per user
13.2t/s
TTFT · p50
165ms
Request success 100% · Included
250 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name sharegpt \
  --dataset-path /tmp/ShareGPT_V3.json \
  --num-prompts 250 \
  --request-rate 0.3 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 08-sharegpt.json
09 · vllm bench serveopen-loop

Reasoning workload

Long chains of thought, 1k in and 4k out, a slow arrival rate of 0.2 because every request costs a lot of decode budget. Tests whether TTFT stays stable.

dataset
random
input / output
1024 / 4096
rate (req/s)
0.2
prompts
50
burstiness
1
Tokens/sec · per user
9.6t/s
TTFT · p50
318ms
Request success 100% · Included
50 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 1024 \
  --random-output-len 4096 \
  --random-range-ratio 0.9 \
  --num-prompts 50 \
  --request-rate 0.2 \
  --burstiness 1.0 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 09-reasoning.json
10 · vllm bench serveopen-loop

Monday morning peak

Random, 4000 in and 500 out, arrival rate 1.5 per second with burstiness 1.0 and at most 25 in parallel. A heavy, independent request stream.

dataset
random
input / output
4000 / 500
rate (req/s)
1.5
prompts
300
burstiness
1
max parallel
25
Tokens/sec · per user
85.5t/s
TTFT · p50
809ms
Request success 100% · Included
300 prompts · seed 42view command →hide command ↑ 
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 300 \
  --request-rate 1.5 \
  --burstiness 1.0 \
  --max-concurrency 25 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 10-monday-peak.json
11 · vllm bench serveopen-loop

Capacity · rate sweep

Six increasing arrival rates, from 0.1 to 1.0 requests per second. At each rung we check whether p95 TTFT still stays under the threshold.

dataset
random
input / output
4000 / 500
rates (req/s)
0.1 · 0.2 · 0.3 · 0.5 · 0.7 · 1.0
prompts
100 · 100 · 100 · 125 · 175 · 250
burstiness
0.7
Req/s · TTFT < 5s
0.8req/s
Req/s · TTFT < 10s
0.8req/s
Request success 100% · Included
6 rates · seed 42view command →hide command ↑ 
# 0.1 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 100 \
  --request-rate 0.1 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.1.json

# 0.2 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 100 \
  --request-rate 0.2 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.2.json

# 0.3 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 100 \
  --request-rate 0.3 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.3.json

# 0.5 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 125 \
  --request-rate 0.5 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.5.json

# 0.7 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 175 \
  --request-rate 0.7 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-0.7.json

# 1.0 req/s
docker exec vllm-bench vllm bench serve \
  --backend openai-chat \
  --base-url http://localhost:8000 \
  --endpoint /v1/chat/completions \
  --model google/gemma-4-E4B-it \
  --tokenizer google/gemma-4-E4B-it \
  --served-model-name gemma-4-e4b-it-bf16 \
  --dataset-name random \
  --random-input-len 4000 \
  --random-output-len 500 \
  --random-range-ratio 0.9 \
  --num-prompts 250 \
  --request-rate 1.0 \
  --burstiness 0.7 \
  --percentile-metrics ttft,tpot,itl,e2el \
  --metric-percentiles 50,90,95,99 \
  --seed 42 \
  --save-result \
  --result-dir /tmp \
  --result-filename 11-rate-sweep-1.0.json
05Notes on this run

Chat

Chat with ten concurrent requests

With 1024 input and 1024 output tokens at ten concurrent requests I measure an average of 21.47 tokens/s per user. The average wait for the first token is 1.05 seconds.

Long context

Wait at 25k context

With 25000 input tokens and 256 output tokens at ten concurrent requests the average wait for the first token is 29.43 seconds. After that the model generates an average of 7.94 tokens/s per user. The measurement alone does not show whether memory, prompt processing or scheduling causes the delay.

Peak load

Requests and wait under peak load

The run handles 0.53 requests/s at a configured arrival rate of 1.5 requests/s. 300 of 300 requests succeed. The p95 wait for the first token is 2.02 seconds.

Measurement coverage

Which measurements count

11 of 11 tests have a measurement that counts under the current Arena rules. The sanity check passed. This is a check on a short generated answer, not a substantive evaluation of the benchmark tasks.

More numbers?
Read the full article.

RunSource data and metadata→WeightsHugging Face→BackTo the arena→
MarkdownSource JSON
Explanation