Statistics: each cell is the MEAN (arithmetic average of scored samples) ± STANDARD ERROR of the mean (SE = sample standard deviation / √n). Scores are LLM-judge quality 0–1 (higher = better); TPS = output tokens/second (higher = faster). Grouped bars of audio transcription quality means per model and clip length. LFM Q8 and LFM bf16 have no long (30 min) data: all 30-min clips failed at the candidate stage (marked n/a).
Quality / ratings
Model
Short (1 min)
Medium (5 min)
Long (30 min)
Gemma E4B BF16 + BF16 mmproj
0.88 ± 0.073
0.80 ± 0.10
0.73 ± 0.033
Gemma E4B UD-IQ2 + BF16 mmproj
0.86 ± 0.075
0.74 ± 0.040
0.33 ± 0.088
LFM Omni Q8 + Q8 mmproj
0.84 ± 0.093
0.85 ± 0.065
failed
LFM Omni bf16 + bf16 mmproj
0.88 ± 0.058
0.78 ± 0.095
failed
Nemotron-3.5-Bolt + dflash + mmproj
0.94 ± 0.060
0.73 ± 0.033
0.50 ± 0.20
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
Short (1 min)
0.88 ± 0.073
0.86 ± 0.075
0.84 ± 0.093
0.88 ± 0.058
0.94 ± 0.060
Medium (5 min)
0.80 ± 0.10
0.74 ± 0.040
0.85 ± 0.065
0.78 ± 0.095
0.73 ± 0.033
Long (30 min)
0.73 ± 0.033
0.33 ± 0.088
failed
failed
0.50 ± 0.20
Tokens per second (TPS)
Model
Short (1 min)
Medium (5 min)
Long (30 min)
Gemma E4B BF16 + BF16 mmproj
110 ± 20
110 ± 11
47 ± 8.1
Gemma E4B UD-IQ2 + BF16 mmproj
100 ± 22
110 ± 6.1
53 ± 18
LFM Omni Q8 + Q8 mmproj
110 ± 19
130 ± 2.4
failed
LFM Omni bf16 + bf16 mmproj
66 ± 11
81 ± 2.1
failed
Nemotron-3.5-Bolt + dflash + mmproj
120 ± 21
78 ± 1.1
47 ± 7.6
Model
Gemma Q8
Gemma IQ2M
LFM Q8
LFM bf16
Nemotron
Short (1 min)
110 ± 20
100 ± 22
110 ± 19
66 ± 11
120 ± 21
Medium (5 min)
110 ± 11
110 ± 6.1
130 ± 2.4
81 ± 2.1
78 ± 1.1
Long (30 min)
47 ± 8.1
53 ± 18
failed
failed
47 ± 7.6
Generated from results-*/results.json via html/build_report.py → html/aggregate.json on 2026-11-12.
