Gemma E4B BF16 + BF16 mmproj

Quality & speed (tokens/s) — LLM-judge scores 0–1; cells show mean ± standard error (SE)

Statistics: each cell is the MEAN (arithmetic average of scored samples) ± STANDARD ERROR of the mean (SE = sample standard deviation / √n); (n/N) = scored/total samples. Scores are LLM-judge quality 0–1 (higher = better); TPS = output tokens/second (higher = faster).

Quality / ratings

Image understanding

Metric	Score (0–1)
Analysis	0.87 ± 0.026
Total question-answer	0.66 ± 0.20
Public transit timetable reading	0.64 ± 0.19

Summarization

Metric	Score (0–1)
English	0.71 ± 0.028

Translation

Combined (both directions)

Metric	Score (0–1)
English (all en→X)	0.73 ± 0.015
African / Middle eastern	0.77 ± 0.028
Western European	0.74 ± 0.015
Eastern European	0.71 ± 0.032
Asian	0.73 ± 0.021
All pairs	0.74

To English (X→en)

Metric	Score (0–1)
African / Middle eastern	0.78 ± 0.039
Western European	0.72 ± 0.019
Eastern European	0.76 ± 0.048
Asian	0.75 ± 0.040
All X→en	0.74 ± 0.015

From English (en→X)

Metric	Score (0–1)
African / Middle eastern	0.75 ± 0.034
Western European	0.76 ± 0.025
Eastern European	0.66 ± 0.038
Asian	0.72 ± 0.026
All en→X	0.73 ± 0.015

Audio transcription

Metric	Score (0–1)
Short (1 min)	0.88 ± 0.073
Medium (5 min)	0.80 ± 0.10
Long (30 min)	0.73 ± 0.033

Tokens per second (TPS) — average: 150