Build real-time AI translation in under 10 minutes

Glot translates a live conversation as it happens. Your client streams microphone audio into a room, a translator joins that room, and it publishes two things back: translated audio you play, and transcript frames you render as captions. This page describes the streaming model end to end — the lifecycle of a session, how audio flows, and the exact wire format of the transcript. It is the concept guide behind the Translation Playground, which is a working reference client for everything below.

Audio is carried over WebRTC by LiveKit, so you connect with a LiveKit client SDK using the url and token the Glot API returns. You never talk to LiveKit’s API directly — Glot provisions the room and mints the tokens.

How a session works

The moving parts

Room

A live translation session, and also a billing record. Created with POST /v1/rooms. A room has no languages of its own — you can create it before you know who will join or what they speak. The response carries the room id (used to look up usage and to close the room) and the room name (used to mint tokens).

Participant

One connection to a room. Every participant, including the first, enters with a join token from POST /v1/rooms/{room_name}/token, which is where languages are chosen — one participant at a time. Tokens are short-lived and bound to the identity they were issued for; a participant is disconnected when their token expires.

Languages

ISO 639-1 codes, from GET /v1/supported-languages. A participant declares the languages they speak and want to hear. Send two when a single connection covers both sides of the conversation — two colleagues sharing one phone, or a test client like the playground — and one when the speaker has a device to themselves. That count is also what sets the room’s streaming mode. A room translates between at most two languages; a token request that would introduce a third returns 409.

Translator

Joins the room on its own once the room holds two languages, and leaves when fewer than two remain. You never dispatch or dismiss it. It publishes an audio track and transcript frames. It bills at a higher per-minute rate than a participant, which is why it is not present in an idle room.

Streaming modes: mono and dual

Every room has a streaming mode, which says how its audio reaches the translator. It follows from one question: are both speakers on the same device, or does each have their own? The mode decides how many lanes the room’s audio travels on — a lane being one uplink stream of mixed audio going to the translator.

mono — one shared device

Two people, one phone or laptop. Their voices are already mixed by the device’s mic, so the room has one lane carrying both languages.

dual — one device each

Each speaker joins from their own device, wherever they are. The room has one lane per language, so the two sides are never mixed together.

	`mono`	`dual`
Use it when	Two colleagues share one phone, or a room shares one laptop	Each speaker is on their own device
Connections	One, covering both languages	One per language
Languages per join token	Two — `["en", "zh"]`	One — `["en"]` for one side, `["zh"]` for the other
Lanes to the translator	One, carrying both voices	One per language, routed by the token’s language

Both modes translate in both directions, publish audio the same way, and emit the same transcript frames. What differs is how the audio is carried — which is why the mode has to be consistent across a room’s tokens. A mono lane is expected to contain both voices; a dual lane is expected to contain one language’s speakers and nobody else.

A dual room is not limited to two participants — only to two distinct languages. Three people, two speaking English and one speaking Chinese, is a valid dual room: the two English devices are mixed together into the English lane.

Set the mode, or let Glot infer it

Declare it
Let Glot infer it

Pass streaming_mode when you create the room. Use this when you already know the shape of the call.

cURL

curl -X POST https://api.staging.glot.com/v1/rooms \
  -H "Authorization: Bearer $GLOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"streaming_mode": "dual"}'

The room comes back with the mode it recorded, and from then on a join token asking for the wrong number of languages is an error rather than a room that quietly means something else.

JSON

{
  "id": "3f1c…",
  "room": "room-9a2b41c7d5e0",
  "url": "wss://…",
  "streaming_mode": "dual"
}

Omit streaming_mode and the room starts with none. Its first join token then decides, by how many languages it asks for:

two languages on one token → mono
one language → dual

cURL

# This token makes the room dual for the rest of its life.
curl -X POST https://api.staging.glot.com/v1/rooms/room-9a2b41c7d5e0/token \
  -H "Authorization: Bearer $GLOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"identity": "alice", "languages": ["en"]}'

Use this when the client minting the first token is the one that knows how the call will be set up. The room’s response carries "streaming_mode": null until that token lands.

Either way, the mode is fixed for the room’s lifetime — there is no call that changes it — and GET /v1/rooms/{room_id} reports whatever the room settled on.

Every later token must match

Once a room has a mode, a join token whose language count disagrees is refused with 409. The message names the mode and the count it expects:

409 Conflict

This room is in dual mode: each participant joins from their own device,
so a join token must request exactly 1 language.

So a dual room takes exactly one language per token, and a mono room takes two on every connection.

An inferred mode is fixed by the first token even if nobody ever connects with it. Mint a one-language token by mistake against a room with no mode, and that room is dual for good — your only remedy is a new room. Declaring streaming_mode at creation turns that mistake into a 409 on the token instead.

Start a session

Create the room

Languages are not part of this call. Pass room_name to choose your own name — it must be unique among rooms that are currently live — or omit it and one is generated. Pass streaming_mode to pin the room to mono or dual, or omit it and let the first join token decide.

cURL

curl -X POST https://api.staging.glot.com/v1/rooms \
  -H "Authorization: Bearer $GLOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{}'

JSON

{
  "id": "3f1c…",
  "room": "room-9a2b41c7d5e0",
  "url": "wss://…",
  "streaming_mode": null
}

The room needs credit: a request against an organization whose balance is at or below zero returns 402. A call already in progress is never cut off, so a long room can run the balance negative.

Mint a join token

Use the room name from the previous response, not the name you asked for — when you omit room_name the API generates one.One token per connection. The example below carries both languages, which is the mono shape — a shared device. For a dual room, mint one token per speaker with a single language each. See streaming modes.

cURL

curl -X POST https://api.staging.glot.com/v1/rooms/room-9a2b41c7d5e0/token \
  -H "Authorization: Bearer $GLOT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"identity": "alice", "languages": ["en", "zh"]}'

JSON

{
  "room": "room-9a2b41c7d5e0",
  "token": "eyJ…",
  "url": "wss://…",
  "languages": ["en", "zh"]
}

A join token is a live credential for a translation session. Mint it on your server and hand only the token to the browser — never ship your API key to a client.

Connect and publish the microphone

Nothing is translated until audio arrives, and the translator only joins once the room holds two languages.

TypeScript

import { Room } from "livekit-client"

const room = new Room()
await room.connect(url, token)
await room.localParticipant.setMicrophoneEnabled(true)

Close the room when the call ends

Addressed by the room’s id — the persisted UUID — not the name tokens are minted against.

cURL

curl -X DELETE https://api.staging.glot.com/v1/rooms/3f1c… \
  -H "Authorization: Bearer $GLOT_API_KEY"

Metering is per participant, and we make a best effort to stop it as each one disconnects — so leaving the room does not silently run up minutes. But the room itself stays open until you close it or it ends on its own, so close it explicitly rather than relying on either. Closing an already-ended room is a no-op that still returns 204, so a retry on the way out is safe. The usage record survives — a room is a billing record and outlives the call it measured.

Play the translated audio

The translator publishes one audio track per language in the room, as ordinary remote tracks. Which of them a client attaches depends on the room’s streaming mode.

mono — one shared device
dual — one device each

The shared device wants all of them: both speakers are listening to the same speaker grille, in both directions. So attach every remote audio track you get subscribed to, and detach it when it goes away.

TypeScript

import { RoomEvent, Track, type RemoteTrack } from "livekit-client"

const container = document.querySelector("#glot-audio")!

room
  .on(RoomEvent.TrackSubscribed, (track: RemoteTrack) => {
    if (track.kind !== Track.Kind.Audio) return
    container.appendChild(track.attach())
  })
  .on(RoomEvent.TrackUnsubscribed, (track: RemoteTrack) => {
    track.detach().forEach((el) => el.remove())
  })

There is no other human connection to worry about — the one device carries both speakers, so everything remote in the room came from the translator.

Attaching everything plays the wrong things here. Each client has to attach only the translation into its own language — the language its token was minted with — for two reasons:

The other language’s track carries this speaker’s own words translated. That track is what the other device plays.
The other participants are separate connections now, so their raw microphones are remote tracks too. Attaching those plays untranslated speech over the top of the translation.

So filter on two things: the track comes from the translator, and it carries your language.

TypeScript

import {
  ParticipantKind,
  RoomEvent,
  Track,
  type RemoteParticipant,
  type RemoteTrack,
  type RemoteTrackPublication,
} from "livekit-client"

/** The language this client joined in — the one it wants to hear. */
const MY_LANGUAGE = "en"

/**
 * The translator names each track it publishes after the language that track carries,
 * and it spells the language the way the model does rather than as an ISO 639-1 code.
 * So the code you sent to `/token` has to be mapped before it can be matched here.
 */
const TRACK_NAME_BY_LANGUAGE: Record<string, string> = {
  en: "English",
  zh: "Chinese",
}

const container = document.querySelector("#glot-audio")!

room
  .on(
    RoomEvent.TrackSubscribed,
    (
      track: RemoteTrack,
      publication: RemoteTrackPublication,
      participant: RemoteParticipant,
    ) => {
      if (track.kind !== Track.Kind.Audio) return
      // Human mics reach this handler too — theirs is the speech the translator is
      // translating, and playing it would talk over the translation.
      if (participant.kind !== ParticipantKind.AGENT) return
      if (publication.trackName !== TRACK_NAME_BY_LANGUAGE[MY_LANGUAGE]) return
      container.appendChild(track.attach())
    },
  )
  .on(RoomEvent.TrackUnsubscribed, (track: RemoteTrack) => {
    track.detach().forEach((el) => el.remove())
  })

Filtering at attach time still pays to receive audio you throw away. To decline it on the wire instead, turn auto-subscribe off and opt in per publication — each one is announced before you subscribe to it:

TypeScript

// Nothing is subscribed until asked for, so `TrackPublished` is the decision point.
await room.connect(url, token, { autoSubscribe: false })

room.on(RoomEvent.TrackPublished, (publication: RemoteTrackPublication) => {
  publication.setSubscribed(
    publication.trackName === TRACK_NAME_BY_LANGUAGE[MY_LANGUAGE],
  )
})

In either mode the elements can live in a hidden container — they are playback sinks, not UI. Browsers block autoplay until the user has interacted with the page, so connect from a click rather than on page load.

Live transcription

The translator publishes one JSON frame per text stream on the topic glot.transcript, over the same room connection. There is no separate transcription endpoint or socket to open. An utterance arrives in two forms — a stream of partial chunks for liveness, then one authoritative segment frame that supersedes them. Streaming Events documents both shapes, the parser, and the fold; this section is only how you attach them to a room.

This is deliberately not LiveKit’s own lk.transcription topic. That convention assumes the sender is the participant being transcribed, and here the sender is a translator relaying somebody else’s speech — so LiveKit’s built-in transcription events never see these frames. Register a handler for glot.transcript yourself.

Reading frames off the topic

Register the handler before you connect, so no frame published in the first moments of the session arrives without one.

TypeScript

import { Room } from "livekit-client"

const TRANSCRIPT_TOPIC = "glot.transcript"

const room = new Room()
let entries: TranscriptEntry[] = []

// Reads are chained, not run concurrently. A text-stream handler returns void, so the
// SDK does not await it — two overlapping readAll() promises can settle out of order and
// interleave one utterance's chunks into another's. Chaining costs nothing here: each
// frame is one small, self-contained stream.
let reads = Promise.resolve()

room.registerTextStreamHandler(TRANSCRIPT_TOPIC, (reader) => {
  reads = reads
    .then(async () => {
      const frame = parseTranscriptFrame(await reader.readAll())
      if (frame === null) return
      entries = applyTranscriptFrame(entries, frame)
      render(entries)
    })
    .catch((err) => {
      // An unreadable frame costs one caption; a rejection left on the chain would
      // poison every read after it.
      console.error("Failed to read a transcript frame", err)
    })
})

await room.connect(url, token)

Registering a second handler for the same topic on the same Room throws. Build a fresh Room per session and register once.

parseTranscriptFrame and applyTranscriptFrame are in Streaming Events — the parser is total and never throws, and the fold is what turns partials and segments into the caption list rendered below.

Rendering captions

The playground’s transcript panel is a good default, and each of its choices comes from a property of the stream:

One row per utterance, keyed by segment id — the translation as the headline, the speech it came from underneath once it settles.
Label the direction per row, as EN → ZH. Both directions land in one list, and with two Latin-script languages the text itself won’t say which way round a line goes. Until the segment frame lands the source is unknown, so render … → ZH and fill the left side in.
Dim text that isn’t final and show a cursor while it is streaming, so a live caption reads as provisional.
Autoscroll only while the reader is at the live edge. Somebody who has scrolled up to re-read something must not be dragged back down by the next utterance.
Announce new rows with role="log" and aria-live="polite", and set aria-busy on rows that are still streaming so a screen reader can tell a growing caption from a settled one.

System messages

The translator also publishes what it is doing — dialling its model, retrying it, losing it — on a second topic, glot.system. Without it, a model outage reaches your user as translated audio that simply stops, which is indistinguishable from nobody talking. See Streaming Events for the message catalog, the parsing rules, and why a handler on that topic alone is not enough.

Pitfalls

Nothing is translated and no translator appears

The room holds fewer than two languages. Either a second participant has to join with a different language, or one connection must mint its token with two — {"languages": ["en", "zh"]}. A single-language room stays monolingual and untranslated.

A token request returns 409

Three different causes, and the message says which:

The room name is already in use by a live room (on POST /v1/rooms).
The token’s language count contradicts the room’s streaming mode — one language in a mono room, or two in a dual one.
The token would introduce a third language. This check is against who is connected right now, so once a speaker leaves, their language frees up and a different one is admitted.

A room ended up in the wrong streaming mode

Its first join token inferred it. A room with no declared mode takes the mode implied by that token’s language count, whether or not anyone connects with it, and nothing changes it afterwards. Create a new room, and pass streaming_mode at creation so the mode comes from your intent rather than from a token.

A new session opens showing the last call's captions

Clear the transcript explicitly when the token changes. A fresh token is a fresh session, and state that outlives the connection won’t reset itself.

Captions duplicate themselves, or chunks interleave

Both are fold and read-ordering bugs rather than room problems — see the Streaming Events pitfalls.

Translation went quiet and nothing said why

The audio stopping is all the transcript topic can tell you. Subscribe to system messages to distinguish a model that is reconnecting from a room where nobody is speaking.

The room is still open after everyone left

Disconnecting participants stops their metering — best effort, as each one leaves — but it does not end the room. Call DELETE /v1/rooms/{room_id} when the call is over rather than leaving the room to end on its own.

See it working

Translation Playground, in the Glot dashboard, runs this whole flow in the browser — create a room, connect a mic, and watch partial and segment frames land in the live transcript.

​How a session works

​The moving parts

​Streaming modes: mono and dual

mono — one shared device

dual — one device each

​Set the mode, or let Glot infer it

​Every later token must match

​Start a session

​Play the translated audio

​Live transcription

​Reading frames off the topic

​Rendering captions

​System messages

​Pitfalls

See it working

How a session works

The moving parts

Streaming modes: mono and dual

Set the mode, or let Glot infer it

Every later token must match

Start a session

Play the translated audio

Live transcription

Reading frames off the topic

Rendering captions

System messages

Pitfalls