Statistics: each cell is the MEAN (arithmetic average of scored samples) ± STANDARD ERROR of the mean (SE = sample standard deviation / √n). Scores are LLM-judge quality 0–1 (higher = better); TPS = output tokens/second (higher = faster). Grouped bars of image understanding quality means per model.
Quality / ratings
Model
Analysis
Total question-answer
Public transit timetable reading
Gemma E4B BF16 + BF16 mmproj
0.87 ± 0.026
0.66 ± 0.20
0.64 ± 0.19
Gemma E4B UD-IQ2 + BF16 mmproj
0.79 ± 0.038
0.26 ± 0.19
0.22 ± 0.16
LFM Omni Q8 + Q8 mmproj
0.75 ± 0.053
0.40 ± 0.24
0.30 ± 0.20
LFM Omni bf16 + bf16 mmproj
0.72 ± 0.041
0.40 ± 0.24
0.36 ± 0.22
Nemotron-3.5-Bolt + dflash + mmproj
0.82 ± 0.030
0.62 ± 0.22
0.56 ± 0.20
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
Analysis
0.87 ± 0.026
0.79 ± 0.038
0.75 ± 0.053
0.72 ± 0.041
0.82 ± 0.030
Total question-answer
0.66 ± 0.20
0.26 ± 0.19
0.40 ± 0.24
0.40 ± 0.24
0.62 ± 0.22
Public transit timetable reading
0.64 ± 0.19
0.22 ± 0.16
0.30 ± 0.20
0.36 ± 0.22
0.56 ± 0.20
Tokens per second (TPS)
Model
Analysis
Total question-answer
Public transit timetable reading
Gemma E4B BF16 + BF16 mmproj
110 ± 4.2
100 ± 16
100 ± 16
Gemma E4B UD-IQ2 + BF16 mmproj
130 ± 7.8
130 ± 15
130 ± 14
LFM Omni Q8 + Q8 mmproj
150 ± 3.8
22 ± 9.4*
48 ± 25*
LFM Omni bf16 + bf16 mmproj
87 ± 2.7
8.0 ± 3.6*
23 ± 14*
Nemotron-3.5-Bolt + dflash + mmproj
110 ± 9.9
150 ± 22
140 ± 23
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
Analysis
110 ± 4.2
130 ± 7.8
150 ± 3.8
87 ± 2.7
110 ± 9.9
Total question-answer
100 ± 16
130 ± 15
22 ± 9.4*
8.0 ± 3.6*
150 ± 22
Public transit timetable reading
100 ± 16
130 ± 14
48 ± 25*
23 ± 14*
140 ± 23
TPS for image question/answer and timetable reading may be inaccurate because the result text is very short.
Generated from results-*/results.json via html/build_report.py → html/aggregate.json on 2026-11-12.
