> ## Documentation Index > Fetch the complete documentation index at: https://docs.glot.com/llms.txt > Use this file to discover all available pages before exploring further. # Streaming Events Schema > Every frame the translator publishes into a room — transcript captions and system messages — with their formats, parsing rules, and how to fold them into state Alongside the translated audio, the translator publishes JSON frames into the room over LiveKit **text streams**. There is no second connection to open and no separate endpoint to poll: if you are connected to the room, you are already receiving them. This page is the wire reference for those frames — what each one means, its exact shape, and the rules for reading it. For creating a room and connecting to it, see [Build real-time AI translation in under 10 minutes](/translation_and_streaming). ## Two topics Frames are separated by topic, and the split is not cosmetic — the two carry different kinds of truth and are read with different rules. **What is being said.** Live chunks of each translation, then the confirmed utterance in both languages. This is what you render as captions. **What the translator is doing.** Dialling its model, retrying it, losing it. This is what keeps a stalled translation from looking like silence. Both are plain JSON, broadcast to everyone in the room, and both carry a `v` field so you can recognize a frame you cannot read. These are deliberately **not** LiveKit's own `lk.transcription` topic. That convention assumes the sender is the participant being transcribed, and here the sender is a translator relaying somebody else's speech — so LiveKit's built-in transcription events never see these frames. Register handlers for `glot.transcript` and `glot.system` yourself. ## Transcript frames The model streams a translation before it confirms it, so an utterance arrives in two forms. Both carry a `segment` id, which is what ties them together. One chunk of the translation as it is produced, for liveness only. ```json title="JSON" theme={null} { "v": 1, "kind": "partial", "segment": "seg_18", "language": "zh", "text": "我很好, ", "last": false } ``` The chunk **verbatim** — a delta, not a running total. Accumulate deltas per `segment` for the typing effect. Marks the partial that ended the utterance. It means "this stopped growing", **not** "this is complete" — it exists so you can stop a cursor even when the `segment` frame never arrives. The complete utterance, in both languages, published once the model confirms it. ```json title="JSON" theme={null} { "v": 1, "kind": "segment", "segment": "seg_18", "language": "zh", "text": "我很好，谢谢。", "source_language": "en", "source_text": "I'm good, thanks." } ``` The authoritative translation. It **supersedes** every `partial` sharing this `segment` id, so replace your accumulation rather than extending it. ISO 639-1 code of the language the speech came from. Only a `segment` frame carries direction — a partial tells you the language it is arriving *in* and nothing about where it came from. What was actually spoken, in the source language. Both kinds share: Frame-shape version, currently `1`. Groups the partials of one utterance with the `segment` frame that supersedes them. ISO 639-1 code of the language this text was translated **into**. ## System messages The translator runs on a model it reaches over the network, and when that model is slow to answer or drops, **the translated audio simply stops** — which from your user's side is indistinguishable from nobody talking. These messages are how you tell those two apart. | `kind` | `severity` | Means | | -------------------- | ---------- | ---------------------------------------------------------------------------- | | `model.connecting` | `info` | Dialling the model. Nothing is being translated yet. | | `model.ready` | `info` | Streaming. Translated audio is live. | | `model.reconnecting` | `warning` | The connection dropped and is being retried. Translation is paused. | | `model.recovered` | `info` | Back, on a **new** session. See the warning below. | | `model.unavailable` | `error` | The model could never be reached. The translator leaves. | | `model.lost` | `error` | The model went away mid-call and retries are exhausted. Translation is over. | ```json title="JSON" theme={null} { "v": 1, "kind": "model.reconnecting", "severity": "warning", "attempt": 2, "retry_in_seconds": 1.46 } ``` `model.reconnecting` is the only kind carrying extra fields: How many times in a row the connection has failed. It resets once a connection survives long enough to be considered healthy, so it says "how bad is this right now", not "how many times has this call reconnected". Seconds until the next attempt. **Relative, never a timestamp** — your clock and the translator's do not agree — so count down from the delta. Every message carries: Envelope version, currently `1`. Dot-namespaced as `subject.happening`. The list above will grow. `info`, `warning`, or `error` — so a kind you have never heard of is still rankable. A reconnect starts a **fresh session on the model**, which has no memory of the utterance that was in flight when the connection died. Whatever was mid-sentence is never translated and no late caption is coming for it — that is a caption left permanently at `stopped`. Treat `model.recovered` as a resumption, not a catch-up. Short interruptions are deliberately not announced. A connection that was healthy and comes straight back costs the room about a second, and firing `model.reconnecting` for that would flash a warning nobody needed. Expect messages for outages worth reporting, not for every blip. ## Versions and unknown kinds The translator and your client deploy independently, so both topics carry `v` — but what `v` covers differs, and so does what you do with a `kind` you don't recognize. | | `glot.transcript` | `glot.system` | | -------------- | ------------------------------------------------------------------------------ | ----------------------------------------------------- | | `v` covers | The whole frame shape | The envelope only | | Bumped when | Any breaking change to a frame | The envelope changes — **not** when a `kind` is added | | Unknown `v` | Drop the frame | Drop the frame | | Unknown `kind` | **Drop it** — there are only two, and a third would be a shape you cannot fold | **Keep it** — new kinds ship without a `v` bump | That asymmetry is the one thing to carry away from this page. A system parser that only admits the kinds it knows today will silently swallow every message added after your last release; a transcript parser that admits an unknown kind has nothing sensible to do with it. ## Reading frames off a topic Register handlers **before** you connect, so nothing published in the first moments of the session arrives without one. The quick start's [Reading frames off the topic](/translation_and_streaming#reading-frames-off-the-topic) shows the transcript handler in full — the system topic is registered on the same `Room`, the same way. Reads are chained rather than run concurrently. A text-stream handler returns `void`, so the SDK does not await it — two overlapping `readAll()` promises can settle out of order and interleave one utterance's chunks into another's. What is specific to having two topics is that each needs **its own chain**. Sharing the transcript's would queue "the model is reconnecting" behind whatever captions are already in flight, and that is precisely the moment the message matters. ```ts title="TypeScript" theme={null} const SYSTEM_TOPIC = "glot.system" let systemState: SystemState = { model: null, attempt: null, retryInSeconds: null } // A chain of its own, deliberately not the transcript's. let systemReads = Promise.resolve() room.registerTextStreamHandler(SYSTEM_TOPIC, (reader) => { systemReads = systemReads .then(async () => { const message = parseSystemMessage(await reader.readAll()) if (message === null) return systemState = applySystemMessage(systemState, message) renderStatus(systemState) }) .catch((err) => { // A rejection left on the chain would poison every read after it. console.error("Failed to read a system message", err) }) }) ``` A `Room` throws if you register twice for the **same** topic — but `glot.transcript` and `glot.system` are different topics, so both handlers sit happily on one `Room`. Register each once, before `room.connect()`. ## Parsing Parsing runs on network input inside a stream handler, where a throw takes out your read chain rather than surfacing anywhere useful. Both parsers below are total: they return `null` for anything they cannot read, and never raise. Treat malformed JSON, a non-object, an unknown `kind`, and a future `v` all as "not a frame I can read". ```ts title="TypeScript" theme={null} const TRANSCRIPT_VERSION = 1 export function parseTranscriptFrame(payload: string): TranscriptFrame | null { let raw: unknown try { raw = JSON.parse(payload) } catch { return null } if (typeof raw !== "object" || raw === null) return null const { v, kind, segment, language, text } = raw as Record if (v !== TRANSCRIPT_VERSION) return null if (typeof segment !== "string" || typeof language !== "string" || typeof text !== "string") { return null } if (kind === "partial") { return { kind, segment, language, text, last: (raw as { last?: unknown }).last === true } } if (kind === "segment") { const { source_language: sourceLanguage, source_text: sourceText } = raw as Record // Both halves of the pair or nothing: half a pair renders as a caption that silently // drops the direction it claims to show. if (typeof sourceLanguage !== "string" || typeof sourceText !== "string") return null return { kind, segment, language, text, sourceLanguage, sourceText } } return null } ``` Validate the envelope strictly and the `kind` loosely — the opposite of the transcript parser, for the reason in [versions and unknown kinds](#versions-and-unknown-kinds). ```ts title="TypeScript" theme={null} const SYSTEM_VERSION = 1 type SystemMessage = { kind: string severity: "info" | "warning" | "error" attempt: number | null retryInSeconds: number | null } export function parseSystemMessage(payload: string): SystemMessage | null { let raw: unknown try { raw = JSON.parse(payload) } catch { return null } if (typeof raw !== "object" || raw === null) return null const { v, kind, severity, attempt, retry_in_seconds: retryInSeconds } = raw as Record if (v !== SYSTEM_VERSION) return null if (typeof kind !== "string") return null if (severity !== "info" && severity !== "warning" && severity !== "error") return null const num = (value: unknown) => typeof value === "number" && Number.isFinite(value) ? value : null // `kind` is kept whatever it is — a message you don't recognize is still one you // can rank by severity, and dropping it here is how you miss the next one we add. return { kind, severity, attempt: num(attempt), retryInSeconds: num(retryInSeconds) } } ``` ## Folding into state The two topics fold differently, and the difference follows from what they describe. A transcript is a **growing list** of utterances; the system state is **one connection**, so its messages collapse onto each other. ### Transcript: extend, then replace A partial **extends** its utterance, a segment **replaces** it. Appending a segment's text would duplicate the whole utterance, because the segment carries what the partials already spelled out. ```ts title="TypeScript" theme={null} type TranscriptStatus = "streaming" | "stopped" | "final" type TranscriptEntry = { segment: string /** Language translated into. */ language: string /** Accumulated while streaming, replaced wholesale when confirmed. */ text: string /** Known only once the utterance is confirmed. */ sourceLanguage: string | null sourceText: string | null status: TranscriptStatus } export function applyTranscriptFrame( entries: TranscriptEntry[], frame: TranscriptFrame, ): TranscriptEntry[] { const index = entries.findIndex((entry) => entry.segment === frame.segment) if (frame.kind === "segment") { const confirmed: TranscriptEntry = { segment: frame.segment, language: frame.language, text: frame.text, sourceLanguage: frame.sourceLanguage, sourceText: frame.sourceText, status: "final", } // Appended when unseen rather than dropped: partials can be lost across a reconnect, // and the confirmed text is the half worth showing. if (index === -1) return [...entries, confirmed] return entries.map((entry, i) => (i === index ? confirmed : entry)) } if (index === -1) { return [ ...entries, { segment: frame.segment, language: frame.language, text: frame.text, sourceLanguage: null, sourceText: null, status: frame.last ? "stopped" : "streaming", }, ] } const current = entries[index] // A partial that arrives after its utterance was confirmed is stale — the frames raced, // or the translator re-sent one. Appending it would tack a fragment onto a finished caption. if (current.status === "final") return entries return entries.map((entry, i) => i === index ? { ...current, text: current.text + frame.text, status: frame.last ? "stopped" : "streaming" } : entry, ) } ``` The three statuses are genuinely different states, not one flag: | Status | Meaning | Reached by | | ----------- | ------------------------------------- | -------------------------------- | | `streaming` | Still growing. Show a cursor. | Any `partial` with `last: false` | | `stopped` | Stopped growing, but never confirmed. | A `partial` with `last: true` | | `final` | The authoritative text landed. | A `segment` frame | An utterance interrupted by a model reconnect stops without ever being confirmed, so it stays `stopped` forever. A UI that treated `stopped` as done would claim text was confirmed when it never was. ### System: collapse onto the newest These messages describe where **one** connection is, so a second `model.reconnecting` replaces the first. They arrive in order, so the newest wins outright. ```ts title="TypeScript" theme={null} type ModelStatus = "connecting" | "ready" | "reconnecting" | "unavailable" | "lost" type SystemState = { /** `null` until the translator has said anything — including before it joins. */ model: ModelStatus | null attempt: number | null retryInSeconds: number | null } const MODEL_STATUS_BY_KIND: Record = { "model.connecting": "connecting", "model.ready": "ready", "model.reconnecting": "reconnecting", "model.unavailable": "unavailable", "model.lost": "lost", } export function applySystemMessage( state: SystemState, message: SystemMessage, ): SystemState { // Recovery is not a status of its own — being back is what `ready` already means. if (message.kind === "model.recovered") { return { model: "ready", attempt: null, retryInSeconds: null } } const model = MODEL_STATUS_BY_KIND[message.kind] // A message outside the model's lifecycle. You may still want to surface it by // severity, but it does not move this state machine. if (model === undefined) return state // Only the retry carries these, so every other transition clears them — a stale // countdown under a status that has moved on reads as a bug. return { model, attempt: model === "reconnecting" ? message.attempt : null, retryInSeconds: model === "reconnecting" ? message.retryInSeconds : null, } } ``` ## Messages tell you what happened, attributes tell you what is true This is the part to get right, and it is why a `glot.system` handler alone is not enough. A text stream is **not replayed** to a participant who joins after it was published, and the two terminal messages — `model.unavailable` and `model.lost` — are followed immediately by the translator leaving the room, so they can lose the race and never arrive at all. The translator therefore also keeps its current state on its **participant attribute** `status`, which *is* replayed to late joiners and is set before it leaves: | `status` | Meaning | | ------------ | ---------------------------------------------------- | | `connecting` | Joined, dialling the model. | | `active` | Translating. | | `error` | The model is gone. Translation has stopped for good. | | `closing` | The translator is shutting down. | So: drive your UI from the messages, and use `status === "error"` as the authority that overrides them. Without that fallback a client can sit on "reconnecting…" forever for a translator that has already given up and left. ```ts title="TypeScript" theme={null} import { ParticipantKind, RoomEvent } from "livekit-client" const syncTranslatorStatus = () => { for (const participant of room.remoteParticipants.values()) { if (participant.kind !== ParticipantKind.AGENT) continue if (participant.attributes.status !== "error") continue systemState = { model: "lost", attempt: null, retryInSeconds: null } renderStatus(systemState) } } room .on(RoomEvent.ParticipantAttributesChanged, syncTranslatorStatus) // A translator already in an error state when you join never fires a change event. .on(RoomEvent.ParticipantConnected, syncTranslatorStatus) ``` Call it once more after `room.connect()` resolves, for the case where you joined a session already in progress. ## Showing the translator's status Captions have their own guidance in [Rendering captions](/translation_and_streaming#rendering-captions). For the system state: * **Say nothing when `model` is `ready` or `null`.** A working translator needs no commentary, and a badge that is always present stops being read. * **Pair it with, not instead of, your connection status.** They fail independently: your WebRTC connection can be perfectly healthy while the translator has lost its model, and "Connected" alone would then claim translation is working. * **Render the countdown only while there is one.** `retryInSeconds` can be absent or effectively zero, and "retrying in 0s" reads as broken. Round up — a countdown showing `1s` for 1.4 seconds finishes before the retry does. * **Announce it with `aria-live="polite"`.** This narrates something the user cannot see happening — audio that has gone quiet — so a screen reader gets no signal otherwise. ## Pitfalls You are appending `segment` text to the accumulated partials. A segment replaces, it never extends. Your reads run concurrently. Chain them per topic, as shown above. The model reconnected. That starts a new session, which never receives the rest of the utterance — so its caption stays at `stopped` and no `segment` frame is coming. That is what `stopped` is for; don't render it as confirmed. A `model.reconnecting` message is how you know this is what happened. You are driving it from `glot.system` alone. `model.lost` is published as the translator shuts down and can lose that race, so it is not guaranteed to arrive. Read the translator's `status` attribute as well and treat `error` as authoritative — see [messages and attributes](#messages-tell-you-what-happened-attributes-tell-you-what-is-true). Expected. A connection that drops and comes straight back is not announced, because flashing a warning for a one-second gap is worse than staying quiet. Messages cover outages worth reporting, not every interruption. Your parser is rejecting unknown `kind`s. New system kinds ship without a `v` bump, by design — keep any frame whose envelope is valid and ignore the ones your UI has no use for. See [versions and unknown kinds](#versions-and-unknown-kinds). Clear both the transcript and the system state when the token changes. A fresh token is a fresh session, and state that outlives the connection won't reset itself. **Translation Playground**, in the Glot dashboard, renders both topics side by side — watch partial and segment frames land in the live transcript, and the translator's status change beside them.