Gemma E4B UD-IQ2 + BF16 mmproj

Quality & speed (tokens/s) — LLM-judge scores 0–1; cells show mean ± standard error (SE)

Statistics: each cell is the MEAN (arithmetic average of scored samples) ± STANDARD ERROR of the mean (SE = sample standard deviation / √n); (n/N) = scored/total samples. Scores are LLM-judge quality 0–1 (higher = better); TPS = output tokens/second (higher = faster).

Quality / ratings

Image understanding

Metric	Score (0–1)
Analysis	0.79 ± 0.038
Total question-answer	0.26 ± 0.19
Public transit timetable reading	0.22 ± 0.16

Summarization

Metric	Score (0–1)
English	0.60 ± 0.026

Translation

Combined (both directions)

Metric	Score (0–1)
English (all en→X)	0.62 ± 0.017
African / Middle eastern	0.72 ± 0.039
Western European	0.66 ± 0.017
Eastern European	0.52 ± 0.047
Asian	0.64 ± 0.021
All pairs	0.64

To English (X→en)

Metric	Score (0–1)
African / Middle eastern	0.75 ± 0.053
Western European	0.65 ± 0.024
Eastern European	0.56 ± 0.075
Asian	0.69 ± 0.043
All X→en	0.66 ± 0.021

From English (en→X)

Metric	Score (0–1)
African / Middle eastern	0.65 ± 0.043
Western European	0.67 ± 0.022
Eastern European	0.48 ± 0.057
Asian	0.62 ± 0.024
All en→X	0.62 ± 0.017

Audio transcription

Metric	Score (0–1)
Short (1 min)	0.86 ± 0.075
Medium (5 min)	0.74 ± 0.040
Long (30 min)	0.33 ± 0.088

Tokens per second (TPS) — average: 190