> ## Documentation Index
> Fetch the complete documentation index at: https://docs.geko.sh/llms.txt
> Use this file to discover all available pages before exploring further.

# live streaming transcription

> transcribe speech as it is spoken over a websocket. partial results while someone is still talking, finals you can keep.

Send audio while it is being spoken and get transcripts back before the speaker finishes. Built for the case where waiting is not an option — a voice agent that has to answer, or live captions.

<Warning>
  **If you have a file, don't use this.** [`transcribe()`](/stt) is more accurate on long audio — the model sees whole utterances instead of a growing tail — and costs **a third as much**. Streaming exists for live audio, not for convenience.
</Warning>

## The endpoint

```
wss://geko--seta-stream-streamer-api.modal.run/v1/stream
```

Authenticate with the same `sk-tokay-…` key as everything else, either as an `Authorization: Bearer …` header or — since browsers cannot set headers on a WebSocket — as a `?key=` query parameter.

Audio must be **PCM16-LE, mono, 16 kHz**. Nothing is resampled server-side: guessing the sample rate of a raw byte stream is how audio ends up silently transcribed at the wrong pitch. Resample before you send.

## TypeScript

```ts theme={null}
import { Geko } from "@gekoai/sdk";

const geko = new Geko({ apiKey: process.env.GEKO_API_KEY });
const stream = await geko.stt.stream({ endOfUtteranceMs: 200 }); // optional, see below

// Read results in one place…
(async () => {
  for await (const event of stream) {
    if (event.type === "partial") process.stdout.write(`\r${event.text}`);
    if (event.type === "final") console.log(`\n${event.text}`);
    if (event.type === "done") console.log(`billed ${event.credits_charged} credits`);
  }
})();

// …push audio from another.
for await (const chunk of microphone) stream.send(chunk);
stream.stop();
```

<Note>
  Node 20 has no global `WebSocket` — it stabilised in Node 22. On Node 20 either upgrade or pass one in: `geko.stt.stream({ webSocket: (await import("ws")).WebSocket })`. The SDK itself stays dependency-free.
</Note>

## Python

There is no Python SDK for streaming yet; the wire protocol is small enough to use directly with [`websockets`](https://pypi.org/project/websockets/):

```python theme={null}
import asyncio, json, os
from websockets.asyncio.client import connect  # pip install websockets

URL = "wss://geko--seta-stream-streamer-api.modal.run/v1/stream?end_of_utterance_ms=200"

async def main():
    headers = {"Authorization": f"Bearer {os.environ['GEKO_API_KEY']}"}
    async with connect(URL, additional_headers=headers,
                       open_timeout=60, max_size=None) as ws:
        print(json.loads(await ws.recv()))            # {"type": "ready", ...}

        async def send_audio():
            async for frame in microphone():          # yours: PCM16-LE mono 16 kHz bytes
                await ws.send(frame)
            await ws.send(json.dumps({"type": "stop"}))

        async def read_events():
            while True:
                event = json.loads(await ws.recv())
                if event["type"] == "partial":
                    print("…", event["text"])
                elif event["type"] == "final":
                    print("✓", event["text"])         # append these: never revised
                elif event["type"] in ("done", "error"):
                    return event

        _, result = await asyncio.gather(send_audio(), read_events())
        print(result)                                  # text, audio_seconds, credits_charged

asyncio.run(main())
```

## Partials and finals

Two kinds of result arrive, and the difference matters:

| event         | meaning                                                                                                                                                                                                                                                                                                                             |
| ------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `partial`     | a running guess at audio that has **not been committed yet**. It **will** be replaced by a later partial or final. Render it; never store it.                                                                                                                                                                                       |
| `final`       | a committed span of transcript, with `start` and `end` in seconds from the start of the stream and a **`confidence`** in \[0, 1] — the model's own posterior over the text. **Never revised.** Append these and you have the transcript; threshold the confidence for "could you repeat that?" behaviour.                           |
| `end_of_turn` | **the human stopped talking.** A `final` fires at any pause; this fires only when the silence *persists* past the turn gate (default 1s), and never fires if speech resumed inside the window. It carries every final committed since the previous turn event. A voice agent should **answer on this** and render on the other two. |
| `done`        | sent after `stop`, carrying the full text, the audio duration, and the credits charged.                                                                                                                                                                                                                                             |

A `final` is emitted when the speaker stops — **however short the reply was**. A one-word "да" finalizes exactly like a full sentence. Committing is also what keeps the cost bounded: once a span is committed it is never re-examined, so a ten-minute call costs no more per minute than a ten-second one.

**How quickly they arrive.** A new `partial` is emitted every **0.25s** of speech. A `final` fires once the speaker has been silent for the **end-of-utterance threshold**, plus roughly 50ms of decode — the decode itself is flat and jitter-free, because every input shape the model can see is pre-warmed when a container boots. Set the threshold yourself (see below) and one value applies at every utterance length: at 200ms a final leaves the server \~250ms after the last word, one-word replies included, and measured end-to-end from Kazakhstan over the public internet it lands **\~0.4s** after the last word with a worst case near 0.5s. Leave it unset and an accuracy-first adaptive default applies: 350ms, or 700ms for replies under \~2 seconds, a split measured to leave transcription accuracy unchanged. Measured on a warm connection against 16 kHz audio fed at real time; a client that sends larger audio frames cannot receive results faster than it sends, so 250ms frames or smaller are recommended.

Every `partial` and `final` also carries **`audio_to`** — how far into the audio you have sent the transcript has reached, in seconds. Comparing it with how much you have sent shows whether the transcript is keeping up with the speech, which lets an agent decide when it is safe to act **without waiting for the end-of-utterance call at all**.

## Tuning end of utterance

The threshold is yours to set, per connection — it is the wait between your caller going quiet and their words arriving as a `final`, so it sits directly in your response time:

```ts theme={null}
// at connect
const stream = await geko.stt.stream({ endOfUtteranceMs: 200 });

// or mid-call, e.g. after discovering the line is noisy
stream.setEndOfUtterance(400);
```

Without the SDK: `?end_of_utterance_ms=200` on the connection URL, or send `{"type":"config","end_of_utterance_ms":200}` at any time — the server acknowledges with the value that took effect.

The server clamps to **100–2000ms** and echoes the value that took effect in `ready`, so you never have to assume. Setting it is also a statement about your audio: one threshold then applies at **every** utterance length. Go low (150–250ms) if your audio is conversational replies and you handle false endpoints downstream — a breath mid-sentence may then split a phrase across two finals, which concatenate to the same transcript but each decode with less context. Go high if you would rather never split a slow, deliberate speaker. If you don't set it, the adaptive default above applies — tuned for accuracy on long-form speech, at the cost of one-word replies taking \~750ms instead of \~250ms.

We measured the low-threshold trade so you don't have to guess: on long-form **read** speech (FLEURS Kazakh), a uniform 200ms threshold costs about **+1.9pp WER** versus the adaptive default, entirely from mid-sentence splits. On short conversational replies the effect is negligible — a reply that fits between two breaths is a single segment either way. If your callers speak in sentences, not monologues, low thresholds are cheap; if they dictate paragraphs, keep the default.

## Turns: when to answer

`end_of_turn_ms` (query param, config message, or `endOfTurnMs` in the SDK) is the second
timer, and it answers a different question from the end-of-utterance threshold. A `final`
means "this transcript is committed" — it fires at a pause, and a pause mid-thought looks
identical to a finished thought at that horizon. `end_of_turn` means "the human stopped":
it fires only when the silence has **persisted** (default 1000ms, clamped 300–5000, `0`
disables), measured from the same clock — the end of speech. If the speaker resumes
inside the window, no event fires and the turn keeps accumulating, which is exactly the
false-endpoint insurance an agent otherwise builds client-side. Set it to your
answer-readiness: \~800–1200ms reads natural in conversation.

## Context: tell the decoder what this call is about

Pass the entities likely to occur — brands, tariff names, streets — and the decoder
favours them when the audio is genuinely ambiguous:

```
wss://…/v1/stream?context=halyk,kaspi,журавлева
{"type":"config","context":["dx5 mix","алатау"]}     ← replaceable mid-call
```

Up to 200 terms, applied to finals. Digit-bearing terms are expanded automatically into
their spoken forms in both languages — send `DX5` as-is and the decoder is primed for
"дэ икс пять" *and* "дэ икс бес". `ready` (or the config ack) reports `context_terms`
accepted and names any `context_skipped` — terms with no expressible spoken form at all.
Biasing is subtractive: a half-matched term earns nothing, so an irrelevant term list
measurably does not bend transcripts (that invariant is part of our release gates).

## Numbers as digits

Transcripts are verbatim by default — the model spells numbers out, because that is what
was said. If the text feeds an LLM or a CRM, ask for digits:

```
wss://…/v1/stream?numerals=digits          → "тариф за 5500 тенге в месяц"
{"type":"config","numerals":"words"}       → switchable mid-call, acknowledged
```

Deterministic inverse normalization for Kazakh and Russian cardinals, applied to every
emitted text (partials, finals, turns, `done`). Deliberately conservative: a lone small
number word stays a word ("ни один клиент" never becomes "ни 1 клиент"), and the
converter provably inverts the exact normalizers our training references use.

## Wire protocol

If you are not using the SDK:

```
client →  connect with Authorization: Bearer sk-tokay-…   (or ?key=…)
          optionally ?end_of_utterance_ms=200&end_of_turn_ms=1000&context=halyk,kaspi
server →  {"type":"ready","model":"seta-kk-ru-v2","sample_rate":16000,
           "end_of_utterance_ms":200,          ← echoed only when you set it
           "end_of_turn_ms":1000,              ← always announced; 0 = disabled
           "context_terms":2}                  ← plus context_skipped when relevant
client →  binary frames: PCM16-LE, mono, 16 kHz
server →  {"type":"partial","text":"…","audio_to":12.3}
server →  {"type":"final","text":"…","start":12.5,"end":15.8,"audio_to":16.1,"confidence":0.94}
server →  {"type":"end_of_turn","text":"…","start":12.5,"end":15.8,"audio_to":16.9}
client →  {"type":"config","end_of_utterance_ms":200,"end_of_turn_ms":800,"context":["…"]}
client →  {"type":"stop"}
server →  {"type":"done","text":"…","audio_seconds":90.5,"credits_charged":543}
server →  {"type":"error","message":"…"}
```

An invalid or missing key is refused at the handshake with **HTTP 403**, before the socket opens.

When the **server** ends the session it closes with code `1000` and a reason you can act on: `stop` (you asked), `idle` (no audio for 30s — reconnect when you have some), or `session_limit` (25 minutes — reconnect to continue). A close without one of these reasons is the network, not us.

## Cold starts

The service scales to zero, so the first connection after an idle period waits while a container comes up. That boot is a **GPU memory snapshot restore** — the model is already loaded inside the checkpoint — so it takes **\~15 seconds**, and the container re-warms its decode paths before accepting the socket: the first request on a fresh container behaves exactly like the thousandth. Connections after that open in well under a second, and the first partial arrives about 1s in — a little behind the steady-state 0.5s cadence, because the opening moments of a stream rarely contain enough speech to decode into words yet.

Three consequences worth designing for:

* **Set a generous handshake timeout.** Many WebSocket clients default to 10s, which is not enough to survive a cold start. The SDK defaults to 60s and exposes `openTimeout`.
* **Don't park an idle "spare" socket to dodge the cold start.** It doesn't work — a socket with no audio is closed after 30s — and it holds GPU capacity that other calls could be using. Connect when the call starts; a warm handshake is sub-second.
* **For a demo or a call window, ask us to pin a warm container.** It removes the cold start entirely. [Get in touch](mailto:amirlan@geko.sh).

## Pricing

\*\*6 credits per audio-second — $0.864 per hour of audio**, against $0.36/hour for file transcription. Billed on audio **accepted**, once, when the socket closes — including if your client disconnects without sending `stop`.

```ts theme={null}
geko.stt.estimateStreamCredits(3600); // 21600 credits === $0.864
```

The difference is a cost difference, not a surcharge: see [why streaming costs more](/billing#why-streaming-costs-more).

## Limits

* One socket is one call. A session may run up to **25 minutes**, then closes cleanly with reason `session_limit` — reconnect to continue.
* A socket with **no audio for 30s** is closed with reason `idle`. Silence during a call is fine — keep sending it as frames; the timeout is for sockets nobody is feeding.
* Audio not committed before you disconnect is still billed — the server accepted it.
* Transcripts are lowercase and unpunctuated, as with file transcription: the model's output vocabulary contains neither. See [accuracy](/stt/accuracy).