Skip to main content
Every number on this page was measured by us on the model we serve, unless explicitly marked published. The evaluation harness and the exact commands live in the geko-seta repo, and the spec sheet is generated from JSON rather than typed by hand.

​
headline

95% confidence intervals: KSC2 [8.43, 8.98], FLEURS kk [7.56, 9.19], FLEURS ru [6.78, 8.59]. WER is computed corpus-level (total errors ÷ total reference words), not as the mean of per-utterance rates — the latter flatters a model on short clips.

​
versus Whisper large-v3

Identical audio, same harness: Whisper also degrades worse under telephony conditions (+12.3% relative, against our +5.7%). Our telephony number beats Whisper’s clean number by 4.4×.
The KSC2 rows for Whisper use a 1,200-clip subset for cost control. At a 5× gap, that makes no material difference.

​
versus ISSAI (published)

ISSAI’s KSC2 paper (Interspeech 2022, Table 2) reports a Transformer streaming recogniser trained on KSC2, evaluated on the same official test split we use: Their system is streaming and ours is offline, and streaming inherently costs accuracy — so part of that gap is architectural, not just modelling. Even so, our telephony-degraded 9.41% still beats their clean 15.6% by 40%.

​
per domain

Where the gains actually come from (KSC2 test, WER): Fine-tuning helps spontaneous speech most and costs a little on easy read speech. That trade was deliberate: conversational audio is the product.

​
telephony

Real phone audio is narrowband and codec-crushed, so we derive a telephony copy of each test set — 300–3400 Hz band-limit → 8 kHz → G.711 μ-law 8-bit → back to 16 kHz: Only 5–8% relative degradation, where an unadapted wideband model typically loses 30–80%. The likely reason is exposure: a quarter of the training clips were not natively 16 kHz, and a 300-hour chunk of the corpus is mobile-recorded.
This is simulated telephony. It models codec and bandwidth, but not handset noise, packet loss, automatic gain control, or overlapping speakers. Treat the measured drop as a lower bound on real call-centre audio.

​
code-switching

KSC2 is explicitly a Kazakh–Russian code-switching corpus, with intra-word switching common enough that its authors defined suffix-level rules for it. Detection here is precision-first and asymmetric on purpose: Russian’s alphabet is a strict subset of Kazakh’s, so orthography can prove a token is Kazakh (via the exclusive letters әғқңөұүһі) but never that it is Russian. As a control, both monolingual Russian test sets read 0.00% code-switched.
Code-switch counts include established loanwords, so read them as an upper bound on genuine intra-sentential switching.

​
speed and cost

Measured on an A10, which is the best $/audio-hour for this model: 1,000 hours of recordings ≈ 2.65 GPU-hours ≈ **4.10∗∗.Youpay4.10**. You pay 0.36/audio-hour — see billing.

​
what we have not measured

Stated plainly, because it affects how much weight these numbers deserve:
  • No real telephony audio yet. Everything above is clean or simulated.
  • No head-to-head against Google STT v2 or Azure Speech, the only other paid engines that support Kazakh.
  • No conversational Russian test set — Russian is validated on read speech only, so the Russian half of the code-switching claim is weaker than the Kazakh half.
  • Not covered at all: streaming/incremental decoding, speaker diarization, punctuation, truecasing.

​
model card