live streaming transcription

Send audio while it is being spoken and get transcripts back before the speaker finishes. Built for the case where waiting is not an option — a voice agent that has to answer, or live captions.

If you have a file, don’t use this. transcribe() is more accurate on long audio — the model sees whole utterances instead of a growing tail — and costs a third as much. Streaming exists for live audio, not for convenience.

The endpoint

wss://geko--seta-stream-streamer-api.modal.run/v1/stream

Authenticate with the same sk-tokay-… key as everything else, either as an Authorization: Bearer … header or — since browsers cannot set headers on a WebSocket — as a ?key= query parameter. Audio must be PCM16-LE, mono, 16 kHz. Nothing is resampled server-side: guessing the sample rate of a raw byte stream is how audio ends up silently transcribed at the wrong pitch. Resample before you send.

TypeScript

import { Geko } from "@gekoai/sdk";

const geko = new Geko({ apiKey: process.env.GEKO_API_KEY });
const stream = await geko.stt.stream({ endOfUtteranceMs: 200 }); // optional, see below

// Read results in one place…
(async () => {
  for await (const event of stream) {
    if (event.type === "partial") process.stdout.write(`\r${event.text}`);
    if (event.type === "final") console.log(`\n${event.text}`);
    if (event.type === "done") console.log(`billed ${event.credits_charged} credits`);
  }
})();

// …push audio from another.
for await (const chunk of microphone) stream.send(chunk);
stream.stop();

Node 20 has no global WebSocket — it stabilised in Node 22. On Node 20 either upgrade or pass one in: geko.stt.stream({ webSocket: (await import("ws")).WebSocket }). The SDK itself stays dependency-free.

Python

There is no Python SDK for streaming yet; the wire protocol is small enough to use directly with websockets:

import asyncio, json, os
from websockets.asyncio.client import connect  # pip install websockets

URL = "wss://geko--seta-stream-streamer-api.modal.run/v1/stream?end_of_utterance_ms=200"

async def main():
    headers = {"Authorization": f"Bearer {os.environ['GEKO_API_KEY']}"}
    async with connect(URL, additional_headers=headers,
                       open_timeout=60, max_size=None) as ws:
        print(json.loads(await ws.recv()))            # {"type": "ready", ...}

        async def send_audio():
            async for frame in microphone():          # yours: PCM16-LE mono 16 kHz bytes
                await ws.send(frame)
            await ws.send(json.dumps({"type": "stop"}))

        async def read_events():
            while True:
                event = json.loads(await ws.recv())
                if event["type"] == "partial":
                    print("…", event["text"])
                elif event["type"] == "final":
                    print("✓", event["text"])         # append these: never revised
                elif event["type"] in ("done", "error"):
                    return event

        _, result = await asyncio.gather(send_audio(), read_events())
        print(result)                                  # text, audio_seconds, credits_charged

asyncio.run(main())

Partials and finals

Two kinds of result arrive, and the difference matters:

event	meaning
`partial`	a running guess at audio that has not been committed yet. It will be replaced by a later partial or final. Render it; never store it.
`final`	a committed span of transcript, with `start` and `end` in seconds from the start of the stream and a `confidence` in [0, 1] — the model’s own posterior over the text. Never revised. Append these and you have the transcript; threshold the confidence for “could you repeat that?” behaviour.
`end_of_turn`	the human stopped talking. A `final` fires at any pause; this fires only when the silence persists past the turn gate (default 1s), and never fires if speech resumed inside the window. It carries every final committed since the previous turn event. A voice agent should answer on this and render on the other two.
`done`	sent after `stop`, carrying the full text, the audio duration, and the credits charged.

A final is emitted when the speaker stops — however short the reply was. A one-word “да” finalizes exactly like a full sentence. Committing is also what keeps the cost bounded: once a span is committed it is never re-examined, so a ten-minute call costs no more per minute than a ten-second one. How quickly they arrive. A new partial is emitted every 0.25s of speech. A final fires once the speaker has been silent for the end-of-utterance threshold, plus roughly 50ms of decode — the decode itself is flat and jitter-free, because every input shape the model can see is pre-warmed when a container boots. Set the threshold yourself (see below) and one value applies at every utterance length: at 200ms a final leaves the server ~250ms after the last word, one-word replies included, and measured end-to-end from Kazakhstan over the public internet it lands ~0.4s after the last word with a worst case near 0.5s. Leave it unset and an accuracy-first adaptive default applies: 350ms, or 700ms for replies under ~2 seconds, a split measured to leave transcription accuracy unchanged. Measured on a warm connection against 16 kHz audio fed at real time; a client that sends larger audio frames cannot receive results faster than it sends, so 250ms frames or smaller are recommended. Every partial and final also carries audio_to — how far into the audio you have sent the transcript has reached, in seconds. Comparing it with how much you have sent shows whether the transcript is keeping up with the speech, which lets an agent decide when it is safe to act without waiting for the end-of-utterance call at all.

Tuning end of utterance

The threshold is yours to set, per connection — it is the wait between your caller going quiet and their words arriving as a final, so it sits directly in your response time:

// at connect
const stream = await geko.stt.stream({ endOfUtteranceMs: 200 });

// or mid-call, e.g. after discovering the line is noisy
stream.setEndOfUtterance(400);

Without the SDK: ?end_of_utterance_ms=200 on the connection URL, or send {"type":"config","end_of_utterance_ms":200} at any time — the server acknowledges with the value that took effect. The server clamps to 100–2000ms and echoes the value that took effect in ready, so you never have to assume. Setting it is also a statement about your audio: one threshold then applies at every utterance length. Go low (150–250ms) if your audio is conversational replies and you handle false endpoints downstream — a breath mid-sentence may then split a phrase across two finals, which concatenate to the same transcript but each decode with less context. Go high if you would rather never split a slow, deliberate speaker. If you don’t set it, the adaptive default above applies — tuned for accuracy on long-form speech, at the cost of one-word replies taking ~750ms instead of ~250ms. We measured the low-threshold trade so you don’t have to guess: on long-form read speech (FLEURS Kazakh), a uniform 200ms threshold costs about +1.9pp WER versus the adaptive default, entirely from mid-sentence splits. On short conversational replies the effect is negligible — a reply that fits between two breaths is a single segment either way. If your callers speak in sentences, not monologues, low thresholds are cheap; if they dictate paragraphs, keep the default.

Turns: when to answer

end_of_turn_ms (query param, config message, or endOfTurnMs in the SDK) is the second timer, and it answers a different question from the end-of-utterance threshold. A final means “this transcript is committed” — it fires at a pause, and a pause mid-thought looks identical to a finished thought at that horizon. end_of_turn means “the human stopped”: it fires only when the silence has persisted (default 1000ms, clamped 300–5000, 0 disables), measured from the same clock — the end of speech. If the speaker resumes inside the window, no event fires and the turn keeps accumulating, which is exactly the false-endpoint insurance an agent otherwise builds client-side. Set it to your answer-readiness: ~800–1200ms reads natural in conversation.

Context: tell the decoder what this call is about

Pass the entities likely to occur — brands, tariff names, streets — and the decoder favours them when the audio is genuinely ambiguous:

wss://…/v1/stream?context=halyk,kaspi,журавлева
{"type":"config","context":["dx5 mix","алатау"]}     ← replaceable mid-call

Up to 200 terms, applied to finals. Digit-bearing terms are expanded automatically into their spoken forms in both languages — send DX5 as-is and the decoder is primed for “дэ икс пять” and “дэ икс бес”. ready (or the config ack) reports context_terms accepted and names any context_skipped — terms with no expressible spoken form at all. Biasing is subtractive: a half-matched term earns nothing, so an irrelevant term list measurably does not bend transcripts (that invariant is part of our release gates).

Numbers as digits

Transcripts are verbatim by default — the model spells numbers out, because that is what was said. If the text feeds an LLM or a CRM, ask for digits:

wss://…/v1/stream?numerals=digits          → "тариф за 5500 тенге в месяц"
{"type":"config","numerals":"words"}       → switchable mid-call, acknowledged

Deterministic inverse normalization for Kazakh and Russian cardinals, applied to every emitted text (partials, finals, turns, done). Deliberately conservative: a lone small number word stays a word (“ни один клиент” never becomes “ни 1 клиент”), and the converter provably inverts the exact normalizers our training references use.

Wire protocol

If you are not using the SDK:

client →  connect with Authorization: Bearer sk-tokay-…   (or ?key=…)
          optionally ?end_of_utterance_ms=200&end_of_turn_ms=1000&context=halyk,kaspi
server →  {"type":"ready","model":"seta-kk-ru-v2","sample_rate":16000,
           "end_of_utterance_ms":200,          ← echoed only when you set it
           "end_of_turn_ms":1000,              ← always announced; 0 = disabled
           "context_terms":2}                  ← plus context_skipped when relevant
client →  binary frames: PCM16-LE, mono, 16 kHz
server →  {"type":"partial","text":"…","audio_to":12.3}
server →  {"type":"final","text":"…","start":12.5,"end":15.8,"audio_to":16.1,"confidence":0.94}
server →  {"type":"end_of_turn","text":"…","start":12.5,"end":15.8,"audio_to":16.9}
client →  {"type":"config","end_of_utterance_ms":200,"end_of_turn_ms":800,"context":["…"]}
client →  {"type":"stop"}
server →  {"type":"done","text":"…","audio_seconds":90.5,"credits_charged":543}
server →  {"type":"error","message":"…"}

An invalid or missing key is refused at the handshake with HTTP 403, before the socket opens. When the server ends the session it closes with code 1000 and a reason you can act on: stop (you asked), idle (no audio for 30s — reconnect when you have some), or session_limit (25 minutes — reconnect to continue). A close without one of these reasons is the network, not us.

Cold starts

The service scales to zero, so the first connection after an idle period waits while a container comes up. That boot is a GPU memory snapshot restore — the model is already loaded inside the checkpoint — so it takes ~15 seconds, and the container re-warms its decode paths before accepting the socket: the first request on a fresh container behaves exactly like the thousandth. Connections after that open in well under a second, and the first partial arrives about 1s in — a little behind the steady-state 0.5s cadence, because the opening moments of a stream rarely contain enough speech to decode into words yet. Three consequences worth designing for:

Set a generous handshake timeout. Many WebSocket clients default to 10s, which is not enough to survive a cold start. The SDK defaults to 60s and exposes openTimeout.
Don’t park an idle “spare” socket to dodge the cold start. It doesn’t work — a socket with no audio is closed after 30s — and it holds GPU capacity that other calls could be using. Connect when the call starts; a warm handshake is sub-second.
For a demo or a call window, ask us to pin a warm container. It removes the cold start entirely. Get in touch.

Pricing

**6 credits per audio-second —

0.864 per hour of audio**, against

0.36/hour for file transcription. Billed on audio accepted, once, when the socket closes — including if your client disconnects without sending stop.

geko.stt.estimateStreamCredits(3600); // 21600 credits === $0.864

The difference is a cost difference, not a surcharge: see why streaming costs more.

Limits

One socket is one call. A session may run up to 25 minutes, then closes cleanly with reason session_limit — reconnect to continue.
A socket with no audio for 30s is closed with reason idle. Silence during a call is fine — keep sending it as frames; the timeout is for sockets nobody is feeding.
Audio not committed before you disconnect is still billed — the server accepted it.
Transcripts are lowercase and unpunctuated, as with file transcription: the model’s output vocabulary contains neither. See accuracy.

get started

text-to-speech

speech-to-text

guides

sdk

reference

live streaming transcription

The endpoint

TypeScript

Python

Partials and finals

Tuning end of utterance

Turns: when to answer

Context: tell the decoder what this call is about

Numbers as digits

Wire protocol

Cold starts

Pricing

Limits

​The endpoint

​TypeScript

​Python

​Partials and finals

​Tuning end of utterance

​Turns: when to answer

​Context: tell the decoder what this call is about

​Numbers as digits

​Wire protocol

​Cold starts

​Pricing

​Limits

The endpoint

TypeScript

Python

Partials and finals

Tuning end of utterance

Turns: when to answer

Context: tell the decoder what this call is about

Numbers as digits

Wire protocol

Cold starts

Pricing

Limits