Statistics: each cell is the MEAN (arithmetic average of scored samples) ± STANDARD ERROR of the mean (SE = sample standard deviation / √n). Scores are LLM-judge quality 0–1 (higher = better); TPS = output tokens/second (higher = faster). Grouped bars of translation quality means per model and language geography, combined both directions (en-X and X-en). Exact values are in the combined table below.
Quality / ratings
Combined (both directions)
Model
English (all en→X)
African / Middle eastern
Western European
Eastern European
Asian
All pairs
Gemma E4B BF16 + BF16 mmproj
0.73 ± 0.015
0.77 ± 0.028
0.74 ± 0.015
0.71 ± 0.032
0.73 ± 0.021
0.74
Gemma E4B UD-IQ2 + BF16 mmproj
0.62 ± 0.017
0.72 ± 0.039
0.66 ± 0.017
0.52 ± 0.047
0.64 ± 0.021
0.64
LFM Omni Q8 + Q8 mmproj
0.37 ± 0.027
0.36 ± 0.063
0.48 ± 0.029
0.31 ± 0.041
0.32 ± 0.030
0.40
LFM Omni bf16 + bf16 mmproj
0.37 ± 0.027
0.37 ± 0.063
0.49 ± 0.029
0.32 ± 0.042
0.30 ± 0.030
0.40
Nemotron-3.5-Bolt + dflash + mmproj
0.67 ± 0.018
0.73 ± 0.035
0.70 ± 0.017
0.69 ± 0.037
0.66 ± 0.028
0.69
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
English (all en→X)
0.73 ± 0.015
0.62 ± 0.017
0.37 ± 0.027
0.37 ± 0.027
0.67 ± 0.018
African / Middle eastern
0.77 ± 0.028
0.72 ± 0.039
0.36 ± 0.063
0.37 ± 0.063
0.73 ± 0.035
Western European
0.74 ± 0.015
0.66 ± 0.017
0.48 ± 0.029
0.49 ± 0.029
0.70 ± 0.017
Eastern European
0.71 ± 0.032
0.52 ± 0.047
0.31 ± 0.041
0.32 ± 0.042
0.69 ± 0.037
Asian
0.73 ± 0.021
0.64 ± 0.021
0.32 ± 0.030
0.30 ± 0.030
0.66 ± 0.028
All pairs
0.74
0.64
0.40
0.40
0.69
To English (X→en)
Model
African / Middle eastern
Western European
Eastern European
Asian
All X→en
Gemma E4B BF16 + BF16 mmproj
0.78 ± 0.039
0.72 ± 0.019
0.76 ± 0.048
0.75 ± 0.040
0.74 ± 0.015
Gemma E4B UD-IQ2 + BF16 mmproj
0.75 ± 0.053
0.65 ± 0.024
0.56 ± 0.075
0.69 ± 0.043
0.66 ± 0.021
LFM Omni Q8 + Q8 mmproj
0.43 ± 0.071
0.49 ± 0.038
0.27 ± 0.060
0.38 ± 0.060
0.43 ± 0.028
LFM Omni bf16 + bf16 mmproj
0.43 ± 0.075
0.50 ± 0.039
0.27 ± 0.060
0.35 ± 0.061
0.43 ± 0.029
Nemotron-3.5-Bolt + dflash + mmproj
0.72 ± 0.052
0.69 ± 0.025
0.76 ± 0.048
0.73 ± 0.049
0.71 ± 0.019
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
African / Middle eastern
0.78 ± 0.039
0.75 ± 0.053
0.43 ± 0.071
0.43 ± 0.075
0.72 ± 0.052
Western European
0.72 ± 0.019
0.65 ± 0.024
0.49 ± 0.038
0.50 ± 0.039
0.69 ± 0.025
Eastern European
0.76 ± 0.048
0.56 ± 0.075
0.27 ± 0.060
0.27 ± 0.060
0.76 ± 0.048
Asian
0.75 ± 0.040
0.69 ± 0.043
0.38 ± 0.060
0.35 ± 0.061
0.73 ± 0.049
All X→en
0.74 ± 0.015
0.66 ± 0.021
0.43 ± 0.028
0.43 ± 0.029
0.71 ± 0.019
From English (en→X)
Model
African / Middle eastern
Western European
Eastern European
Asian
All en→X
Gemma E4B BF16 + BF16 mmproj
0.75 ± 0.034
0.76 ± 0.025
0.66 ± 0.038
0.72 ± 0.026
0.73 ± 0.015
Gemma E4B UD-IQ2 + BF16 mmproj
0.65 ± 0.043
0.67 ± 0.022
0.48 ± 0.057
0.62 ± 0.024
0.62 ± 0.017
LFM Omni Q8 + Q8 mmproj
0.22 ± 0.11
0.48 ± 0.044
0.36 ± 0.056
0.29 ± 0.035
0.37 ± 0.027
LFM Omni bf16 + bf16 mmproj
0.25 ± 0.11
0.48 ± 0.045
0.38 ± 0.055
0.28 ± 0.033
0.37 ± 0.027
Nemotron-3.5-Bolt + dflash + mmproj
0.75 ± 0.022
0.71 ± 0.024
0.62 ± 0.049
0.64 ± 0.032
0.67 ± 0.018
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
African / Middle eastern
0.75 ± 0.034
0.65 ± 0.043
0.22 ± 0.11
0.25 ± 0.11
0.75 ± 0.022
Western European
0.76 ± 0.025
0.67 ± 0.022
0.48 ± 0.044
0.48 ± 0.045
0.71 ± 0.024
Eastern European
0.66 ± 0.038
0.48 ± 0.057
0.36 ± 0.056
0.38 ± 0.055
0.62 ± 0.049
Asian
0.72 ± 0.026
0.62 ± 0.024
0.29 ± 0.035
0.28 ± 0.033
0.64 ± 0.032
All en→X
0.73 ± 0.015
0.62 ± 0.017
0.37 ± 0.027
0.37 ± 0.027
0.67 ± 0.018
Tokens per second (TPS)
Combined (both directions)
Model
English (all en→X)
African / Middle eastern
Western European
Eastern European
Asian
All pairs
Gemma E4B BF16 + BF16 mmproj
160 ± 1.5
150 ± 3.1
160 ± 1.4
160 ± 1.5
160 ± 2.1
160
Gemma E4B UD-IQ2 + BF16 mmproj
200 ± 2.2
200 ± 5.3
200 ± 1.7
200 ± 2.2
200 ± 2.8
200
LFM Omni Q8 + Q8 mmproj
23 ± 2.8
18 ± 2.7
23 ± 2.8
15 ± 2.4
31 ± 4.2
24
LFM Omni bf16 + bf16 mmproj
96 ± 0.62
95 ± 2.2
97 ± 0.38
97 ± 0.44
97 ± 0.37
97
Nemotron-3.5-Bolt + dflash + mmproj
190 ± 4.8
190 ± 8.0
190 ± 5.7
180 ± 8.1
180 ± 6.8
180
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
English (all en→X)
160 ± 1.5
200 ± 2.2
23 ± 2.8
96 ± 0.62
190 ± 4.8
African / Middle eastern
150 ± 3.1
200 ± 5.3
18 ± 2.7
95 ± 2.2
190 ± 8.0
Western European
160 ± 1.4
200 ± 1.7
23 ± 2.8
97 ± 0.38
190 ± 5.7
Eastern European
160 ± 1.5
200 ± 2.2
15 ± 2.4
97 ± 0.44
180 ± 8.1
Asian
160 ± 2.1
200 ± 2.8
31 ± 4.2
97 ± 0.37
180 ± 6.8
All pairs
160
200
24
97
180
To English (X→en)
Model
African / Middle eastern
Western European
Eastern European
Asian
All X→en
Gemma E4B BF16 + BF16 mmproj
150 ± 2.9
150 ± 1.8
160 ± 1.7
160 ± 3.5
160 ± 1.2
Gemma E4B UD-IQ2 + BF16 mmproj
200 ± 3.0
200 ± 2.2
200 ± 2.9
200 ± 3.1
200 ± 1.4
LFM Omni Q8 + Q8 mmproj
20 ± 3.8
26 ± 3.9
18 ± 4.1
28 ± 3.6
24 ± 2.3
LFM Omni bf16 + bf16 mmproj
97 ± 0.52
96 ± 0.62
97 ± 0.86
97 ± 0.91
97 ± 0.38
Nemotron-3.5-Bolt + dflash + mmproj
190 ± 11
180 ± 7.2
160 ± 9.2
170 ± 14
180 ± 5.0
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
African / Middle eastern
150 ± 2.9
200 ± 3.0
20 ± 3.8
97 ± 0.52
190 ± 11
Western European
150 ± 1.8
200 ± 2.2
26 ± 3.9
96 ± 0.62
180 ± 7.2
Eastern European
160 ± 1.7
200 ± 2.9
18 ± 4.1
97 ± 0.86
160 ± 9.2
Asian
160 ± 3.5
200 ± 3.1
28 ± 3.6
97 ± 0.91
170 ± 14
All X→en
160 ± 1.2
200 ± 1.4
24 ± 2.3
97 ± 0.38
180 ± 5.0
From English (en→X)
Model
African / Middle eastern
Western European
Eastern European
Asian
All en→X
Gemma E4B BF16 + BF16 mmproj
150 ± 7.6
160 ± 2.0
150 ± 2.3
150 ± 2.6
160 ± 1.5
Gemma E4B UD-IQ2 + BF16 mmproj
190 ± 15
200 ± 2.7
200 ± 3.3
200 ± 3.7
200 ± 2.2
LFM Omni Q8 + Q8 mmproj
15 ± 2.9
20 ± 3.8
13 ± 2.3
32 ± 5.8
23 ± 2.8
LFM Omni bf16 + bf16 mmproj
91 ± 6.5
97 ± 0.30
97 ± 0.24
97 ± 0.37
96 ± 0.62
Nemotron-3.5-Bolt + dflash + mmproj
180 ± 11
200 ± 8.3
190 ± 12
190 ± 7.6
190 ± 4.8
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
African / Middle eastern
150 ± 7.6
190 ± 15
15 ± 2.9
91 ± 6.5
180 ± 11
Western European
160 ± 2.0
200 ± 2.7
20 ± 3.8
97 ± 0.30
200 ± 8.3
Eastern European
150 ± 2.3
200 ± 3.3
13 ± 2.3
97 ± 0.24
190 ± 12
Asian
150 ± 2.6
200 ± 3.7
32 ± 5.8
97 ± 0.37
190 ± 7.6
All en→X
160 ± 1.5
200 ± 2.2
23 ± 2.8
96 ± 0.62
190 ± 4.8
Generated from results-*/results.json via html/build_report.py → html/aggregate.json on 2026-11-12.
