# API reference The API serves real-time transcription: create a session over HTTP, stream its audio over a WebSocket. ## Base URL | region | | |---|---| | Europe | `https://eu1.api.gladia.io` | | United States | `https://us1.api.gladia.io` | Use the region of your account ([Regions](/introduction/regions/)). ## Authentication Send your API key in the `x-gladia-key` header ([Authentication](/introduction/authentication/)): ```bash curl https://eu1.api.gladia.io/v2/live/defaults -H "x-gladia-key: $GLADIA_API_KEY" ``` ## Endpoints | | | |---|---| | [`POST /v2/live`](/api-reference/v2/live/init/) | Create a session, get its WebSocket URL | | [WebSocket `/v2/live?token=…`](/api-reference/v2/live/websocket/) | Stream the audio, receive the messages | | [`GET /v2/live/{id}`](/api-reference/v2/live/get/) | A session and its result | | [`GET /v2/live`](/api-reference/v2/live/list/) | Your sessions | | [`GET /v2/live/{id}/file`](/api-reference/v2/live/get-audio/) | A session's recording | | [`DELETE /v2/live/{id}`](/api-reference/v2/live/delete/) | Delete a session | | [`GET /v2/live/defaults`](/api-reference/v2/live/defaults/) | The values of omitted options | Every page has a **Try it** panel that calls the API with your key, from your browser. ## Errors Errors answer with a JSON body: ```json { "statusCode": 400, "timestamp": "2026-10-12T09:04:17.210Z", "path": "/v2/live", "message": "Invalid parameter(s). See validation_errors for more details.", "request_id": "G-1a2b3c4d", "validation_errors": ["sample_rate Input should be 8000, 16000, 32000, 44100 or 48000"] } ``` | status | meaning | |---|---| | `400` | The request is invalid: `validation_errors` says why. | | `401` | No API key, or an unknown one. | | `403` | The key may not do this: its scopes, or the caller's address (`error` gives the reason). | | `404` | No such session for your account. | | `429` | Your account has its maximum number of sessions open ([Limits](/limits/concurrency/)). | | `500` | No model in this region runs the session as asked: its model, its languages and its features together (`no inference endpoint available`), or none has room right now. | | `503` | The service is busy: retry with backoff. | Quote `request_id` when you contact support. ## OpenAPI The whole API is described in [/openapi.json](/openapi.json) (OpenAPI 3.1), including the WebSocket messages (`x-websocket`): generate a client, or import it in your API tool. --- # Audio recordings When your account records its audio, each session's audio is kept with its transcript, in **FLAC** (lossless, about half the size of WAV). Recording is a setting of your account: ask your Gladia contact to turn it on or off. Download it with [`GET /v2/live/{id}/file`](/api-reference/v2/live/get-audio/): ```bash e2e="skip: needs an account that records its audio (tests/e2e/test_recording.py covers the download)" curl -L https://eu1.api.gladia.io/v2/live/$ID/file -H "x-gladia-key: $GLADIA_API_KEY" -o session.flac ``` - The recording is ready about 40 seconds after the session ends; before that, and for sessions not recorded, the answer is `404`. - The answer may be a redirect (`302`) to a signed download link: follow it (`curl -L`). - A `Range` header returns part of the file (`206`), so players can seek. - `GET /v2/live/{id}` tells whether a session was recorded: `file.filename` and `file.source` are set. Recordings are kept for your account's audio retention ([Data retention](/data/retention/)), and deleted with their session. --- # History Every session of your account stays in its history for your [retention](/data/retention/), with its options and transcript. The same history is in Gladia Workspace, under **Transcriptions**. ## List [`GET /v2/live`](/api-reference/v2/live/list/) returns your sessions, newest first: ```bash curl -G https://eu1.api.gladia.io/v2/live -H "x-gladia-key: $GLADIA_API_KEY" \ -d limit=20 -d status=done -d after_date=2026-10-01T00:00:00Z ``` | parameter | what it does | |---|---| | `limit` | Sessions per page, 1 to 100 (20 by default). | | `status` | `done`, `error`, `processing`; repeat it for several. | | `date`, `after_date`, `before_date` | A day, or a time range (ISO 8601). | | `custom_metadata` | Sessions whose [metadata](/features/custom-metadata/) has these values. | Follow `next` to get the following page, until it is `null`: it carries a `cursor` that keeps deep pages fast. `offset` works too, up to 1,000. ## Read [`GET /v2/live/{id}`](/api-reference/v2/live/get/) returns one session: its `status`, its `request_params`, its `custom_metadata` and, once `done`, its `result`: every final utterance, the full transcript, the translations. ## Delete [`DELETE /v2/live/{id}`](/api-reference/v2/live/delete/) deletes a session that has ended, its transcript and its recording, for good (`202`). A running session cannot be deleted (`403`). --- # Data retention | data | kept | how to delete earlier | |---|---|---| | Sessions and transcripts (history) | 90 days by default | [`DELETE /v2/live/{id}`](/api-reference/v2/live/delete/) | | Audio recordings (accounts that record) | 30 days by default | deleted with their session | Retention is set per account: ask your Gladia contact for another duration, from no history at all to several years. With no history, a session's result is still available for a short time after it ends, then deleted. Data stays in the region of the API address your account uses ([Regions](/introduction/regions/)). Callbacks are delivered once ([Callbacks](/live/callbacks/#delivery)): if your server misses one, read the session with `GET /v2/live/{id}` while it is in your history. --- # Custom metadata **The problem.** A transcript is useless if you cannot tie it back to the call, the customer or the agent it came from. `custom_metadata` attaches your identifiers to the session: they come back with every result and callback, and you can search your history with them. **Where it works.** Every session. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Attach your identifiers ```json { "custom_metadata": { "call_id": "c-8812", "agent": "lea" } } ``` ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python custom_metadata.py meeting.wav`. **Python SDK** ```python title="custom_metadata.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "custom_metadata": { "call_id": "c-8812", "agent": "lea" } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) job = client.live_v2().get(session.session_id) # the session, from your history print("metadata:", job.custom_metadata) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="custom_metadata.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, custom_metadata: { call_id: "c-8812", agent: "lea" } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", async () => { const job = await client.liveV2().get(session.sessionId); // the session, from your history console.log("metadata:", job.custom_metadata); process.exit(); }); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "custom_metadata": { "call_id": "c-8812", "agent": "lea" } }' ``` ### 3. Get them back They are in [`GET /v2/live/{id}`](/api-reference/v2/live/get/), in Gladia Workspace next to the transcript, and in the history: ```bash curl -G https://eu1.api.gladia.io/v2/live -H "x-gladia-key: $GLADIA_API_KEY" \ --data-urlencode 'custom_metadata={"agent": "lea"}' ``` The filter returns the sessions whose metadata **contains** the values given: `{"agent": "lea"}` matches `{"call_id": "c-8812", "agent": "lea"}`. ## Limits and gotchas - **A JSON object.** A string or an array is refused (`400`). - **Small.** The metadata travels with the session's configuration in its WebSocket URL, together with the vocabulary and the spelling dictionary: keep it to identifiers. Too big, `POST /v2/live` answers `400` "the session configuration is too large…". - **Not redacted.** [PII redaction](/features/pii-redaction/) applies to transcripts, never to metadata: no names, phone numbers or emails in it. - **Exact values.** The history filter compares values exactly: `"Lea"` does not match `"lea"`. - **Set once.** It cannot be changed after `POST /v2/live`. --- # Custom spelling **The problem.** The model hears a word correctly but writes it its own way: "e mail" where your style guide says "e-mail", "sequel" for "SQL", "gladia" without its capital. Custom spelling rewrites those variants into your spelling, in every transcript, before you receive them. **Where it works.** Every model. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Map each spelling to its variants ```json { "realtime_processing": { "custom_spelling": true, "custom_spelling_config": { "spelling_dictionary": { "SQL": ["sequel"], "e-mail": ["email", "e mail"] } } } } ``` Each **key** is the spelling you want; its **values** are the variants to replace. ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python custom_spelling.py meeting.wav`. **Python SDK** ```python title="custom_spelling.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "realtime_processing": { "custom_spelling": True, "custom_spelling_config": { "spelling_dictionary": { "Gladia": [ "gladya", "glad ia" ], "SQL": [ "sequel" ], "e-mail": [ "email", "e mail" ] } } } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="custom_spelling.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, realtime_processing: { custom_spelling: true, custom_spelling_config: { spelling_dictionary: { Gladia: [ "gladya", "glad ia" ], SQL: [ "sequel" ], "e-mail": [ "email", "e mail" ] } } } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "realtime_processing": { "custom_spelling": true, "custom_spelling_config": { "spelling_dictionary": { "Gladia": [ "gladya", "glad ia" ], "SQL": [ "sequel" ], "e-mail": [ "email", "e mail" ] } } } }' ``` ### 3. Check the result ```json { "type": "transcript", "data": { "is_final": true, "utterance": { "text": "Send the SQL query by e-mail to Gladia." } } } ``` The rewrite applies to partial and final transcripts, and to the text of their words. ## Limits and gotchas - **Whole words, any case.** "sequel" is rewritten, "sequels" is not. Add each form you need as a variant. - **The longest variant wins.** With "e mail" and "mail" both listed, "e mail" is rewritten first. - **It does not change what is recognised.** If the model writes something else entirely, there is nothing to rewrite: help it hear the word with [custom vocabulary](/features/custom-vocabulary/), then fix the spelling here. - **Values must be lists.** `{"SQL": "sequel"}` is refused (`400`): write `{"SQL": ["sequel"]}`. - **Translations are not rewritten.** [Translation](/features/translation/) works from the audio, not from the transcript. - **Size limit.** The spelling dictionary shares the session's URL budget with the vocabulary and `custom_metadata`: a few hundred entries in all, else `400` "the session configuration is too large…". - **Entities keep their tags.** With [entities](/features/named-entity-recognition/) or [PII redaction](/features/pii-redaction/), the rewrite never touches the inline elements, only the words. ## Troubleshooting | symptom | cause | fix | |---|---|---| | a variant is not rewritten | it is part of a longer word, or a different form | add that exact form as a variant | | `400` on `custom_spelling_config` | a value is a string, not a list | wrap it in a list | | the spelling flag is on, nothing changes | `custom_spelling_config` missing, or the variant is never written by the model | send the dictionary; check the raw transcript first | --- # Custom vocabulary **The problem.** Speech models write what is most likely in general. Your product, your customers' names and your jargon are rare words: "Gladia" comes out as "Gladya", "Kubernetes" as "Cuban et ease", "Dr Okafor" as "doctor oak a four". Custom vocabulary tells the model which words to expect, so they come out right the first time. **Where it works.** Every model: `solaria-3`, `solaria-1`, `trantoria-1`. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. List your words The option goes in `realtime_processing` of the session's configuration. Both fields are required: ```json { "realtime_processing": { "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": ["Gladia", "Kubernetes", "Dr Okafor"] } } } ``` ### 2. Start a session with it Each program streams a 16-bit WAV file and prints the final transcripts. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python custom_vocabulary.py meeting.wav`. **Python SDK** ```python title="custom_vocabulary.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": {"languages": ["en"]}, "realtime_processing": { "custom_vocabulary": True, "custom_vocabulary_config": { "vocabulary": [ "Gladia", {"value": "Kubernetes", "pronunciations": ["cuban et ease", "kubernetis"]}, {"value": "Dr Okafor", "intensity": 0.8}, ], "default_intensity": 0.5, }, }, } client = GladiaClient( api_key=os.environ["GLADIA_API_KEY"], api_url=os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io"), ) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="custom_vocabulary.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: ["en"] }, realtime_processing: { custom_vocabulary: true, custom_vocabulary_config: { vocabulary: [ "Gladia", { value: "Kubernetes", pronunciations: ["cuban et ease", "kubernetis"] }, { value: "Dr Okafor", intensity: 0.8 }, ], default_intensity: 0.5, }, }, }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": ["en"] }, "realtime_processing": { "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": [ "Gladia", { "value": "Kubernetes", "pronunciations": ["cuban et ease", "kubernetis"] }, { "value": "Dr Okafor", "intensity": 0.8 } ], "default_intensity": 0.5 } } }' ``` ### 3. Check the result Nothing new arrives: the words simply come out spelled as you listed them, in partial and final transcripts. ```json { "type": "transcript", "data": { "is_final": true, "utterance": { "text": "Dr Okafor deployed Gladia on Kubernetes." } } } ``` ## Options | field | what it does | |---|---| | `vocabulary` | The words or phrases: strings, or objects with their own options. | | `value` | The word or phrase, written exactly as it must appear. | | `pronunciations` | How it may be heard, written as it sounds. Whatever the model writes for one of them is rewritten to `value`. | | `intensity` | How strongly the entry is favoured, from 0 to 1 (`trantoria-1` only, see below). | | `default_intensity` | The intensity of entries that set none. | ## Limits and gotchas - **Fewer, precise entries work better.** List what is specific to your audio. Hundreds of common words, or a whole dictionary, make the model see your words where they were not said. - **Intensity applies to `trantoria-1` only.** `solaria-3` and `solaria-1` read the list as context: every entry counts the same and `intensity` is ignored. On `trantoria-1`, a high intensity replaces similar-sounding words with yours: raise it word by word, and only for the words still missed. - **Size limit.** At most 1,000 entries. The whole session configuration (vocabulary, custom spelling and `custom_metadata` together) must also fit in its WebSocket URL: a few hundred average entries. Beyond, `POST /v2/live` answers `400` with "the session configuration is too large for its WebSocket URL". - **Numbers stay as listed.** With [numbers as digits](/features/numerals/), a vocabulary phrase made of number words ("Seven Eleven") is kept as you wrote it. - **Not a spelling rule.** Vocabulary helps the model *hear* a word. To change how a word is *written* (casing, an abbreviation), use [custom spelling](/features/custom-spelling/). - **One list per session.** The vocabulary cannot be changed while the session runs: create a new session. - **`language` of an entry is not applied**: it is accepted for compatibility with Gladia's cloud API. ## Troubleshooting | symptom | cause | fix | |---|---|---| | `400` "custom_vocabulary_config is required when custom_vocabulary is true" | the flag without the list | send both fields | | `400` "the session configuration is too large…" | the list, the spelling and the metadata do not fit in the URL | shorten the list, keep the words that are actually missed | | your word now appears where it was not said | the list is too long or the intensity too high | remove generic entries, lower `intensity` | | a word is still missed on `solaria-3` | the model hears it differently | add `pronunciations` as it is heard, or fix it with [custom spelling](/features/custom-spelling/) | --- # Named entity recognition **The problem.** A caller gives their name, their order number and a delivery date; your CRM needs those fields, not a paragraph. Named entity recognition finds them in each final transcript as it arrives: people, organizations, places, dates, amounts, emails, phone numbers and about 70 types in all. **Where it works.** Every model. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Turn it on ```json { "named_entity_recognition": true } ``` `realtime_processing.named_entity_recognition: true`, Gladia's field, does the same. ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python entities.py meeting.wav`. **Python SDK** ```python title="entities.py" # pip install gladiaio-sdk requests import os import sys import threading import wave import requests from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "named_entity_recognition": { "entities": [ "name", "organization", "date", "phone_number" ] } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) # `named_entity_recognition` with options is a Gladia Enterprise option the Python SDK does not send yet # (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it. created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10) created.raise_for_status() info = created.json() session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"])) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) elif message.type == "named_entity_recognition": for entity in message.data.results: print(f" {entity.entity_type}: {entity.text}") session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="entities.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, named_entity_recognition: { entities: [ "name", "organization", "date", "phone_number" ] } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `named_entity_recognition` with options is a Gladia Enterprise option not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text_raw); if (message.type === "named_entity_recognition") { for (const entity of message.data.results) console.log(` ${entity.entity_type}: ${entity.text}`); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "named_entity_recognition": { "entities": [ "name", "organization", "date", "phone_number" ] } }' ``` ### 3. Read the entities **The final's text carries them inline**, and `text_raw` keeps the plain sentence: ```json { "type": "transcript", "data": { "id": "00_00000002", "is_final": true, "utterance": { "text": "Hi, this is Marie Curie from Gladia.", "text_raw": "Hi, this is Marie Curie from Gladia.", "entities": [ { "label": "name", "id": 1, "action": "ner", "text": "Marie Curie", "start": 12, "end": 52 }, { "label": "organization", "id": 1, "action": "ner", "text": "Gladia", "start": 58, "end": 102 } ] } } } ``` Right after it, a `named_entity_recognition` message lists them with their times in the audio: ```json { "type": "named_entity_recognition", "error": null, "data": { "utterance_id": "00_00000002", "results": [ { "entity_type": "name", "text": "Marie Curie", "start": 4.9, "end": 5.6, "id": 1, "score": 0.97, "char_start": 12, "char_end": 52 } ] } } ``` ## Options | option | what it does | |---|---| | `entities` | Which types: a preset (`name`, `location`, `date`, `contact`, `financial`, `identifiers`, `pii`…), a list of types and presets, or `{"enable": [...], "disable": [...]}`. Omitted: every type. | | `threshold` | Confidence, from 0 to 1, under which an entity is ignored. Default 0.5. | A parent type (`name`, `location`, `date`) also selects its children (`name_given`, `name_family`, `location_city`…). An unknown type is refused, with the list of known ones. ## Limits and gotchas - **The text changes.** With entities on, `utterance.text` contains `` elements. Display `text_raw`, or render the elements; never show `text` as is to an end user. - **Finals only.** Partials are not tagged. - **`id` follows the entity through the session.** The same person keeps `id` 1, even when only their first name is heard again. `start`/`end` on the utterance and `char_start`/`char_end` in the message locate the element in `text`, not in `text_raw`. - **The message is not stored.** The `named_entity_recognition` messages are not part of the session's result; the `entities` of each utterance are. Store the messages yourself (WebSocket or [callback](/live/callbacks/)) if you need their audio times. - **If tagging fails on a final**, the message comes with `error` set and that final has no words. Rare; the session goes on. - **Personal data stays visible.** Entities are tagged, not hidden: use [PII redaction](/features/pii-redaction/) to remove them. Both together: the types selected for redaction are redacted, the others tagged. ## Troubleshooting | symptom | cause | fix | |---|---|---| | users see `` in your interface | `text` displayed as is | display `text_raw`, or render the elements | | no `named_entity_recognition` message | `messages_config.receive_realtime_processing_events: false` | turn it on | | `rejected: … unknown entity labels` (WebSocket `1011`) | a type that does not exist | use the types and presets of the error message | | too many false entities | low confidence spans | raise `threshold` | --- # Numbers as digits **The problem.** Speech models write numbers as words: "one thousand three hundred fifty five dollars twenty eight cents". Your application needs `$1,355.28` to show an order, fill a form or check a payment. With `numerals`, the transcript arrives with digits. **Where it works.** Every model. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Turn it on ```json { "numerals": true } ``` ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python numerals.py meeting.wav`. **Python SDK** ```python title="numerals.py" # pip install gladiaio-sdk requests import os import sys import threading import wave import requests from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "numerals": True } client = GladiaClient(api_key=API_KEY, api_url=API_URL) # `numerals` is a Gladia Enterprise option the Python SDK does not send yet # (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it. created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10) created.raise_for_status() info = created.json() session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"])) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="numerals.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, numerals: true }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `numerals` is a Gladia Enterprise option not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) { const { text, text_raw } = message.data.utterance; console.log(text, text_raw ? `(spoken: ${text_raw})` : ""); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "numerals": true }' ``` ### 3. Read the result ```json { "type": "transcript", "data": { "is_final": true, "utterance": { "text": "The total is $1,355.28 for 42 items.", "text_raw": "The total is one thousand three hundred fifty five dollars twenty eight cents for forty two items." } } } ``` `text` has the digits; `text_raw` keeps the words as spoken. | spoken | written | |---|---| | "forty two" | `42` | | "one thousand three hundred fifty five dollars twenty eight cents" | `$1,355.28` | | "mille trois cent cinquante-cinq euros vingt-huit" (French) | `1 355,28 €` | | "call me at nine fifteen" | `call me at 9:15` | | "rendez-vous à neuf heures quinze" (French) | `rendez-vous à 9h15` | | "um halb zehn" (German) | `um 9:30` | | "a las nueve y cuarto" (Spanish) | `a las 9:15` | Clock times are written the way the language writes them, with no extra setting. ## Phone numbers and dates Phone numbers are written in their country's format and dates the way the language writes them. Nothing to add: `"numerals": true` is enough wherever the turn router runs (the default), and your messages keep their shape (no turn-router label is added to the finals). | spoken | written | |---|---| | "zéro six douze trente-quatre cinquante-six soixante-dix-huit" (French) | `06 12 34 56 78` | | "two one two five five five zero one two three" | `(212) 555-0123` | | "plus thirty three six twelve thirty four fifty six seventy eight" | `+33 6 12 34 56 78` | | "march twelfth twenty twenty six" | `March 12, 2026` | | "march twelfth twenty twenty six" (with `"region": "GB"`) | `12 March 2026` | | "le douze mars deux mille vingt-six" (French) | `le 12 mars 2026` | | "am zwölften märz" (German) | `am 12. März` | | "el primero de mayo" (Spanish) | `el 1 de mayo` | | "il primo maggio" (Italian) | `il 1º maggio` | **How it works.** The turn router reads each final transcript and tells what its numbers are: an amount, a phone number, a date, a clock time, a count. Numerals then writes each one for what it is: - **a phone number the speaker has finished** is written in the national format of its country. A spoken country code ("plus thirty three") gives the international format. A number that no country's numbering plan knows stays as digits; - **a phone number still being dictated** keeps its digits as said, the pauses' commas removed: it is formatted once the speaker has given all of it; - **a date** is written the language's way (English, French, German, Spanish, Italian, Portuguese, Dutch); - **a clock time, a date, a rank or a phone number** is never written as an amount. **Without the turn router** (`"turn_router": false`, or a host that does not run it), numerals writes numbers, clock times and amounts only: phone numbers and dates stay as plain digits. Digits alone cannot tell a phone number from an order number, an account number or a code, and formatting those as phone numbers would change them. ### The country Phone numbers and dates follow a country. Set it with `region`, an ISO 3166-1 alpha-2 code in capitals: ```json { "numerals": { "region": "CH" } } ``` | `region` | "zéro six douze trente-quatre cinquante-six soixante-dix-huit" | |---|---| | omitted (French) | `06 12 34 56 78` | | `CH` | `061 234 56 78` | Without `region`, the countries where the language is spoken apply, in this order: English US, GB, CA, AU, IE, NZ, IN, ZA; French FR, BE, CH, CA, LU; German DE, AT, CH; Spanish ES, MX, AR, CO, CL, PE; Italian IT, CH; Portuguese PT, BR; Dutch NL, BE. A phone number is formatted for the first of these countries whose numbering plan knows it as dialled. A `region` that is not two capital letters is refused with a 400. ## Limits and gotchas - **The conventions of the language spoken.** Amounts take that language's separators and currency position: set the language (`language_config.languages`) when you know it, or the detected one applies. - **Partials too.** Partial transcripts show digits as well, and may change while the number is still being said. Phone numbers and dates are formatted in the final transcripts only. - **Phone numbers and dates need the turn router.** It runs by default; with `"turn_router": false` they stay as plain digits. - **Numbers said in pieces.** "my card is 4970… 1234…" stays in one utterance thanks to the [number hold](/live/endpointing/#number-hold), on by default. With `turn_router: false`, each piece may come as its own final. - **Vocabulary is kept.** A [custom vocabulary](/features/custom-vocabulary/) phrase made of number words ("Seven Eleven") stays as you listed it. - **Search and storage.** Store `text` for display and `text_raw` if you match on spoken words. - **With PII redaction**, numbers that are personal data (card, phone, account numbers) are redacted after the conversion. ## Troubleshooting | symptom | cause | fix | |---|---|---| | amounts use the wrong separators | another language was detected | set `language_config.languages` | | a long number is split over several finals | `turn_router: false`, or a pause over 2 s | keep the number hold on | | a product name with numbers was converted | it is not in your vocabulary | add it to [custom vocabulary](/features/custom-vocabulary/) | | a phone number or a date stays as plain digits | the turn router is off (`"turn_router": false`) | remove `"turn_router": false` | | a phone number is in another country's format | the language's first country was used | set `numerals.region` | | a phone number stays as digits with the turn router on | no country's numbering plan knows it as dialled, or the speaker had not finished it | check the digits and `numerals.region` | --- # PII redaction **The problem.** Call transcripts end up in a CRM, a data warehouse, an LLM prompt. If they contain card numbers, phone numbers or names, all of those systems now hold personal data, with the compliance work that comes with it. PII redaction removes that data from the final transcripts before you receive them. **Where it works.** Every model. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Turn it on ```json { "pii": true } ``` By default, the `pii` preset is redacted: names, contact details, identity, card and account numbers, dates of birth and the like. ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python pii.py meeting.wav`. **Python SDK** ```python title="pii.py" # pip install gladiaio-sdk requests import os import sys import threading import wave import requests from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "numerals": True, "pii": { "entities": [ "pii", "pci" ], "mode": "marker" } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) # `pii` is a Gladia Enterprise option the Python SDK does not send yet # (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it. created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10) created.raise_for_status() info = created.json() session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"])) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) def on_error(error): if "Unsupported websocket message type" not in str(error): # pii messages: gladiaio/sdk#61 errors.append(error) session.on("error", on_error) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="pii.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, numerals: true, pii: { entities: [ "pii", "pci" ], mode: "marker" } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `pii` is a Gladia Enterprise option not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); if (message.type === "pii") { for (const item of message.data.results) console.log(` redacted ${item.entity_type} #${item.id}`); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "numerals": true, "pii": { "entities": [ "pii", "pci" ], "mode": "marker" } }' ``` ### 3. Read the redacted transcript ```json { "type": "transcript", "data": { "is_final": true, "utterance": { "text": "My card number is and my email .", "words": [{ "word": "", "start": 3.1, "end": 3.4, "confidence": 0.95 }] } } } ``` Right after it, a `pii` message lists what was redacted (WebSocket only): ```json { "type": "pii", "error": null, "data": { "utterance_id": "00_00000005", "results": [ { "entity_type": "credit_card", "id": 1, "action": "marker", "char_start": 18, "char_end": 45 }, { "entity_type": "email_address", "id": 1, "action": "marker", "char_start": 59, "char_end": 88 } ] } } ``` ## Options ```json { "pii": { "entities": ["pii", "pci"], "mode": "mask", "mask_character": "#" } } ``` | option | default | what it does | |---|---|---| | `entities` | `pii` | What to redact: presets (`pii`, `pci`, `phi`, `contact`, `financial`, `name`, `location`…), types, or `{"enable": [...], "disable": [...]}`. | | `mode` | `marker` | `marker`: an element in place of the data (``); `mask`: mask characters inside the element; `tag`: the data kept inside the element (tagging only, nothing hidden). | | `threshold` | `0.5` | Confidence under which a span is not redacted. Lower: more redacted, more false positives. | | `mask_character` | `*` | The character of `mask` mode. | | `coreference` | `heuristics` | How mentions of the same data get the same `id`: `heuristics`, `exact` (same text only) or `off`. | | `return_entities` | `false` | Also return the redacted text in the `pii` message (`text`), for your own vault. | ## Limits and gotchas - **No partial transcripts.** Partials are never redacted, so with redaction on none is sent, on the WebSocket or to callbacks, even with `receive_partial_transcripts: true`. Plan your live display on finals. (`mode: tag` hides nothing and keeps partials.) - **Words lose their text.** A redacted final's words keep their times and speakers, their `word` is empty, and `text_raw` is not sent. - **The audio is not redacted.** If your account records its audio, the recording still contains what was said. Ask for recording to be turned off if that matters to you ([Audio recordings](/data/audio-recordings/)). - **`custom_metadata` is not redacted.** Do not put personal data in it. - **Not with translation.** A session with both `pii` and `translation` is refused when it starts (WebSocket `1011`, `rejected: ...`). - **Finals may come a little later** while a number or an email is still being dictated, so it is redacted whole. - **Detection is statistical.** A rare format, a name spelled letter by letter, or poor audio can slip through, and ordinary words can be redacted. Tune `entities` and `threshold` on your own calls before production. - **`return_entities` brings the data back.** Use it only to store the values somewhere safer than your transcripts. - **The `pii` message is not stored** with the session's result and is not sent to callbacks. ## Troubleshooting | symptom | cause | fix | |---|---|---| | no partial transcripts anymore | redaction on | expected; use finals, or `mode: tag` if you only need tagging | | WebSocket closed `1011` "rejected: … pii redaction" | `translation` also on | drop one of them | | a card number is not redacted | spoken digit by digit with long pauses, or another preset | keep `numerals` and the number hold on, add `pci` | | ordinary words are redacted | low confidence spans | raise `threshold`, narrow `entities` | --- # Punctuation and casing **The problem.** Raw speech recognition gives a flat stream of words: "hello id like to change my address is that possible". Punctuation and casing turn it into sentences people can read, and that downstream tools (summaries, search, LLMs) understand better. **Where it works.** Every model, every language. **On by default**: nothing to do. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ```json { "text": "Hello, I'd like to change my address. Is that possible?" } ``` ## Turning it off To feed your own text processing with the words only, set `punctuation: false`. **Python SDK** ```python title="no_punctuation.py" # pip install gladiaio-sdk requests import os import sys import threading import wave import requests from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "trantoria-1", "language_config": { "languages": [ "en" ] }, "punctuation": False } client = GladiaClient(api_key=API_KEY, api_url=API_URL) # `punctuation` is a Gladia Enterprise option the Python SDK does not send yet # (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it. created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10) created.raise_for_status() info = created.json() session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"])) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="no_punctuation.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "trantoria-1", language_config: { languages: [ "en" ] }, punctuation: false }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `punctuation` is a Gladia Enterprise option not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "trantoria-1", "language_config": { "languages": [ "en" ] }, "punctuation": false }' ``` ## Limits and gotchas - **Finals only.** Punctuation and casing are applied to final transcripts. Partials show the text as the model writes it: their punctuation may differ, then settle in the final. Compare finals with finals. - **Off means off.** With `punctuation: false`, the casing goes too: do not expect sentences. - **A Gladia Enterprise option.** Gladia's cloud API does not know `punctuation`: do not send it there. --- # Sentiment analysis **The problem.** A customer is getting angry and nobody notices before the call goes wrong. Sentiment analysis rates each final sentence as positive, neutral or negative, with its dominant emotion, while the call runs: escalate to a supervisor, flag calls to review, chart a conversation's mood. **Where it works.** Every model. Best in English, French, Spanish, German, Italian, Portuguese, Dutch, Polish, Turkish, Russian, Japanese, Chinese and Arabic. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Turn it on ```json { "realtime_processing": { "sentiment_analysis": true } } ``` ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python sentiment.py meeting.wav`. **Python SDK** ```python title="sentiment.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "realtime_processing": { "sentiment_analysis": True } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) elif message.type == "sentiment_analysis": for result in message.data.results: print(f" {result.sentiment} ({result.emotion})") session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="sentiment.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, realtime_processing: { sentiment_analysis: true } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); if (message.type === "sentiment_analysis") { for (const { sentiment, emotion } of message.data.results) console.log(` ${sentiment} (${emotion})`); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "realtime_processing": { "sentiment_analysis": true } }' ``` ### 3. Read the sentiment After each final comes a `sentiment_analysis` message: ```json { "type": "sentiment_analysis", "error": null, "data": { "utterance_id": "01_00000007", "utterance": { "text": "This is the third time I call about it.", "start": 41.2, "end": 43.6, "language": "en", "channel": 1 }, "results": [ { "sentiment": "negative", "emotion": "anger", "text": "This is the third time I call about it.", "start": 41.2, "end": 43.6, "channel": 1 } ] } } ``` `sentiment` is `positive`, `neutral` or `negative`. `emotion` is one of: adoration, amusement, anger, awe, confusion, contempt, contentment, desire, disappointment, disgust, distress, ecstatic, elation, embarrassment, fear, interest, negative surprise, neutral, pain, positive surprise, realization, relief, sadness, sympathy, triumph (as `snake_case`), or empty. ## Limits and gotchas - **It needs the number hold.** Sentiment comes from the same analysis as the [number hold](/live/endpointing/#number-hold): with `turn_router: false`, no `sentiment_analysis` message is sent, and no error either. - **One sentence at a time.** Each final is rated on its own: a call's overall mood is yours to aggregate. - **Finals only.** Partials are not rated. - **Not stored.** `sentiment_analysis` messages are not part of the session's result: keep them as they arrive, on the WebSocket or with a [callback](/live/callbacks/) (`callback_config.receive_realtime_processing_events`). - **Other languages** get a rating too, less reliable. ## Troubleshooting | symptom | cause | fix | |---|---|---| | no `sentiment_analysis` message | `turn_router: false`, or `receive_realtime_processing_events: false` | remove `turn_router: false`; keep the realtime events on | | sentiment missing in `GET /v2/live/{id}` | not stored with the result | store the messages as they arrive | --- # Speaker diarization **The problem.** A meeting room, a podcast, an interview: several voices on one microphone, one stream of text. You cannot tell what the client said from what the advisor said. Diarization labels each utterance and each word with a `speaker` number. **Where it works.** Every model, on live audio. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Turn it on ```json { "diarization": true } ``` ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python diarization.py meeting.wav`. **Python SDK** ```python title="diarization.py" # pip install gladiaio-sdk requests import os import sys import threading import wave import requests from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "diarization": True, "diarization_config": { "max_speakers": 3 } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) # `diarization` is a Gladia Enterprise option the Python SDK does not send yet # (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it. created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10) created.raise_for_status() info = created.json() session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"])) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: utterance = message.data.utterance print(f"Speaker {utterance.speaker}: {utterance.text}") session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="diarization.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, diarization: true, diarization_config: { max_speakers: 3 } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `diarization` is a Gladia Enterprise option not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) { const { speaker, text } = message.data.utterance; console.log(`Speaker ${speaker}: ${text}`); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "diarization": true, "diarization_config": { "max_speakers": 3 } }' ``` ### 3. Read the speakers ```json { "type": "transcript", "data": { "id": "00_00000004", "is_final": true, "utterance": { "text": "Right. Shall we start?", "start": 31.2, "end": 32.6, "speaker": 1, "channel": 0, "words": [ { "word": "Right.", "start": 31.2, "end": 31.5, "confidence": 0.97, "speaker": 0 }, { "word": "Shall", "start": 31.7, "end": 31.9, "confidence": 0.98, "speaker": 1 }, { "word": "we", "start": 31.9, "end": 32.0, "confidence": 0.99, "speaker": 1 }, { "word": "start?", "start": 32.0, "end": 32.6, "confidence": 0.98, "speaker": 1 } ] } } } ``` **Read the speaker of each word, not the utterance's.** An utterance ends on a pause, not on a change of speaker, so one utterance can hold the words of two people: here speaker 0 says "Right." and speaker 1 asks the question. The utterance's `speaker` is only the one who speaks longest in it. Each word's `speaker` is the label to trust. ### 4. Group the words by speaker To show who said what, cut each final where its words change speaker: ```python def turns(utterance): """The utterance's words as (speaker, text) turns, in order.""" out = [] for word in utterance["words"]: if out and out[-1][0] == word["speaker"]: out[-1] = (word["speaker"], out[-1][1] + " " + word["word"]) else: out.append((word["speaker"], word["word"])) return out ``` A label can also be wrong on one or two words (see the limits below). If short flips bother your display more than lost interjections, merge them client-side: give a run of one or two words the speaker of the runs on both sides of it when both are the same speaker. This also hides real one-word answers ("yes", "okay") from that speaker. ```python def merge_islands(words, max_words=2): """Relabel a run of at most max_words words framed by one same speaker on both sides.""" words = [dict(w) for w in words] runs, start = [], 0 # (first, last + 1) index of each run of one speaker for i in range(1, len(words) + 1): if i == len(words) or words[i]["speaker"] != words[start]["speaker"]: runs.append((start, i)) start = i for (a, _), (b, e), (c, _) in zip(runs, runs[1:], runs[2:]): if e - b <= max_words and words[a]["speaker"] == words[c]["speaker"]: for w in words[b:e]: w["speaker"] = words[a]["speaker"] return words ``` Run it on the words of consecutive finals put together, so a flip on the first word of a final is seen between the previous final and the rest of this one. ## Options | option | what it does | |---|---| | `max_speakers` | The most speakers to tell apart, when you know it. | | `number_of_speakers` | Gladia's name for the same setting. | ## Limits and gotchas - **Up to 4 speakers** are told apart in one session. More voices than that share the 4 numbers. - **Finals arrive a little later**, the time the speakers are known (under a second). Partials are not labelled. - **Numbers, not names.** Speaker numbers are valid within the session only: the same person in two sessions may get another number. Map numbers to people in your application. - **Separate channels are better.** When each person has their own channel (a call recorded in stereo), send the channels instead ([Audio formats](/live/audio-formats/#several-channels)): it is exact, and needs no diarization. - **Overlapping speech** is one utterance: two people talking at once are not split into two. - **One utterance, several speakers.** Utterances are not cut where the speaker changes: a quick reply right after another person spoke often lands in the same final. Read each word's `speaker` (step 3). - **One or two words can flip to another speaker.** The speaker is chosen word by word, and a short word over another voice can take the wrong label. On Gladia's diarization quality study (19 recordings, 1.6 h, 659 finals), the first word's speaker differs from the utterance's `speaker` in 17.3 % of the finals (114). In 55 of them the final really starts with another person's words. In the other 59 the first words carry a wrong label, and in 46 of those the wrong run is one or two words long. Counting every place in a final, there are 0.276 runs of one or two words between two runs of one same speaker per final, real short turns included. - **No smoothing by default.** The inference has an optional server-side smoothing that gives a run of up to two words back to the speaker around it. It is off because it also erases real short turns. On the same study it relabelled 65 runs: 38 wrong labels fixed, 27 real interjections lost. On the meeting recordings, the share of speaker changes found fell from 67.8 % to 59.5 %. Merge short runs client-side (step 4) when you want that trade-off. - `min_speakers` and `enhanced` (Gladia's cloud fields) are accepted and ignored. ## Troubleshooting | symptom | cause | fix | |---|---|---| | everyone is speaker 0 | the voices are very similar, or one microphone picks one voice much louder | use one channel per person when you can | | 5 people, only 4 numbers | the 4-speaker limit | split the session by channel | | finals feel slower with diarization | they wait for their speakers | expected: use partials for live display | | the first word or two of an utterance belong to someone else | the reply starts in the same final, or a short label flip | group by each word's `speaker`; merge one- or two-word runs client-side if you prefer stable turns (step 4) | --- # Translation **The problem.** A support agent in Paris, a customer speaking Spanish; a conference streamed to an international audience. Translation turns each sentence into the languages you choose while the person is still speaking: live subtitles, a bilingual call log, a meeting record in your team's language. **Where it works.** `solaria-3` only. The translation is made **from the audio** of each utterance, not from its transcript. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below. ## Step by step ### 1. Choose the target languages ```json { "model": "solaria-3", "realtime_processing": { "translation": true, "translation_config": { "target_languages": ["en", "fr"] } } } ``` ### 2. Start a session with it Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the `GLADIA_API_KEY` environment variable, then run `python translation.py meeting.wav`. **Python SDK** ```python title="translation.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "es" ] }, "realtime_processing": { "translation": True, "translation_config": { "target_languages": [ "en", "fr" ] } } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) elif message.type == "translation" and message.data.translated_utterance: translated = message.data.translated_utterance print(f" {translated.language}: {translated.text}") session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="translation.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "es" ] }, realtime_processing: { translation: true, translation_config: { target_languages: [ "en", "fr" ] } } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); if (message.type === "translation" && message.data.translated_utterance) { const { language, text } = message.data.translated_utterance; console.log(` ${language}: ${text}`); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "es" ] }, "realtime_processing": { "translation": true, "translation_config": { "target_languages": [ "en", "fr" ] } } }' ``` ### 3. Read the translations Each final transcript is followed by one `translation` message per target language: ```json { "type": "translation", "session_id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "error": null, "data": { "utterance_id": "00_00000003", "utterance": { "text": "¿Dónde está la estación?", "start": 12.4, "end": 13.9, "language": "es", "channel": 0 }, "original_language": "es", "target_language": "en", "translated_utterance": { "text": "Where is the station?", "start": 12.4, "end": 13.9, "language": "en", "channel": 0, "confidence": 1, "words": [] } } } ``` ### 4. Get them all at the end The session's result ([`GET /v2/live/{id}`](/api-reference/v2/live/get/), `post_final_transcript`) has a `translation` block: one result per target language, with its utterances and full text. ## Limits and gotchas - **`solaria-3` only.** With another model, no model can serve the session: `POST /v2/live` answers `500` "no inference endpoint available". - **Only the languages below.** Another code passes the creation, then the session is refused when it starts: the WebSocket closes with code `1011` and the reason `rejected: translation: target language(s) [...] not served`. - **Not with PII redaction.** The translation is made from the audio and would carry the redacted data back: a session with both `pii` and `translation` is refused when it starts (`1011`, `rejected: ...`). - **Finals only.** Partials are not translated. Translations arrive shortly after their final, never before it, and never delay it. - **Same language, same text.** An utterance already in a target language comes back as transcribed. - **Long utterances.** Translation reads up to 60 seconds of an utterance: set `maximum_duration_without_endpointing` (at most 60) for monologues without pauses. - **A translation can fail alone.** It then comes with `error` set and `translated_utterance: null`; the transcript is not affected. Show the original text as a fallback. - **At the end**, the session waits up to 5 seconds for the last translations before `end_session`. - **Spelling and vocabulary** do not apply to translations (they come from the audio). - **Accepted, not applied.** Gladia's cloud fields `model`, `context`, `context_adaptation`, `informal`, `lipsync` and `match_original_utterances` in `translation_config` are accepted for compatibility and have no effect. ## Target languages - Afrikaans (`af`) - Amharic (`am`) - Arabic (`ar`) - Assamese (`as`) - Belarusian (`be`) - Bulgarian (`bg`) - Bangla (`bn`) - Catalan (`ca`) - Danish (`da`) - German (`de`) - English (`en`) - Spanish (`es`) - Persian (`fa`) - French (`fr`) - Galician (`gl`) - Indonesian (`id`) - Italian (`it`) - Japanese (`ja`) - Khmer (`km`) - Korean (`ko`) - Lingala (`ln`) - Lao (`lo`) - Malay (`ms`) - Burmese (`my`) - Nepali (`ne`) - Dutch (`nl`) - Norwegian (`no`) - Pashto (`ps`) - Portuguese (`pt`) - Romanian (`ro`) - Russian (`ru`) - Somali (`so`) - Swedish (`sv`) - Thai (`th`) - Vietnamese (`vi`) - Wolof (`wo`) - Yoruba (`yo`) - Chinese (`zh`) ## Troubleshooting | symptom | cause | fix | |---|---|---| | `500` "no inference endpoint available" at creation | the model is not `solaria-3` | set `"model": "solaria-3"` | | WebSocket closed `1011` "rejected: translation: target language(s)…" | a target not in the list | use the codes above | | WebSocket closed `1011` "rejected: … pii redaction" | `pii` and `translation` together | drop one of them | | no `translation` message | `messages_config.receive_realtime_processing_events: false` | turn it back on, or read the session's result | --- # Welcome The Gladia Enterprise API turns speech into text **while it is spoken**. You open a session, stream audio over a WebSocket, and receive transcripts in a fraction of a second: partial ones while a sentence is spoken, a final one when it ends. On the same connection you can get translations, named entities, redacted personal data, sentiment and end-of-turn events. ## How it works 1. **Create a session** with [`POST /v2/live`](/api-reference/v2/live/init/): its audio format, model, languages and options. The answer has a WebSocket URL. 2. **Stream the audio** to that URL, in small chunks, as it is captured. 3. **Read the messages**: [`transcript`](/api-reference/v2/live/message/transcript/) messages, partial then final, and the messages of the features you turned on. 4. **Stop** with `stop_recording`: the last transcripts arrive, then the whole session's result. The session's transcript stays available afterwards ([`GET /v2/live/{id}`](/api-reference/v2/live/get/)), in your history, and in [Gladia Workspace](/introduction/authentication/). ## What you can build - **Voice agents**: low-latency transcripts and [end-of-turn detection](/live/end-of-turn/), so the agent answers when the caller has finished, not at every pause. - **Call centers**: one speaker per channel, [PII redaction](/features/pii-redaction/), [sentiment](/features/sentiment-analysis/), [named entities](/features/named-entity-recognition/). - **Meetings and live captions**: [speaker diarization](/features/speaker-diarization/), [translation](/features/translation/) into 38 languages, [custom vocabulary](/features/custom-vocabulary/) for names and jargon. ## For LLMs and agents Every page has a Markdown twin (add `.md` to its address), [/llms.txt](/llms.txt) indexes them, [/llms-full.txt](/llms-full.txt) has the whole documentation in one file, and [/openapi.json](/openapi.json) is the OpenAPI document of the API. ## Next - [Quickstart](/introduction/quickstart/): your first transcript. - [Playground](/playground/): try it with your microphone. - [Models](/introduction/models/): pick the right one. --- # Authentication Every HTTP call carries your API key in the `x-gladia-key` header: ```bash curl https://eu1.api.gladia.io/v2/live -H "x-gladia-key: $GLADIA_API_KEY" ``` The WebSocket needs no key: the URL that [`POST /v2/live`](/api-reference/v2/live/init/) returns carries a single-use token for its session. ## API keys Create and revoke keys in **Gladia Workspace**, under **API keys**. Keys start with `ge_` and are shown once, at creation. A revoked key stops working within 30 seconds. A key can be restricted to some products (its **scopes**). The live API needs **Real-time STT**: a key without it gets a `403` with `"error": "api_key_scope_denied"`. A key without any scope reaches every product. ## IP allowlist A workspace can accept API calls from listed addresses only (**Settings → Security**). A call from another address gets a `403` with `"error": "ip_not_allowed"`. The allowlist applies to the API, not to Gladia Workspace itself. ## Keep the key on your server Create sessions from your backend and give browsers and phones only the session's WebSocket URL: it works once, for one session, within 60 seconds, and reveals nothing about your key. The client then streams its audio directly to the API, without going through your servers. ## Errors | status | `error` | meaning | |---|---|---| | `401` | | no key, or an unknown one | | `403` | `api_key_scope_denied` | the key lacks the Real-time STT scope | | `403` | `ip_not_allowed` | the caller's address is not in the workspace's allowlist | Every error body has the same shape: [Errors](/api-reference/#errors). --- # Models Pick a model with `model` in the init body of [`POST /v2/live`](/api-reference/v2/live/init/). | model | choose it for | |---|---| | `solaria-3` | Multilingual conversations, automatic language detection, and [translation](/features/translation/): the only model that translates. | | `solaria-1` | The widest language coverage: 99 languages and Cantonese, detected or set. | | `trantoria-1` | The lowest latency, for voice agents: set the language spoken (about 40, English and Spanish included). | Without `model`, a session runs on your account's default model. ## Features by model - solaria-3: Multilingual, with automatic language detection; the only model that translates. Languages: 29. - solaria-1: Gladia's multilingual model: 99 languages and Cantonese, detected or set. Languages: 99. - trantoria-1: The lowest latency: transcripts within a fraction of a second. About 40 languages, English and Spanish included: set the language spoken. Languages: about 40. | feature | solaria-3 | solaria-1 | trantoria-1 | |---|---|---|---| | Custom vocabulary | yes | yes | yes | | Punctuation and casing | yes | yes | yes | | Numbers as digits | yes | yes | yes | | Named entities | yes | yes | yes | | PII redaction | yes | yes | yes | | End of turn | yes | yes | yes | | Number hold | yes | yes | yes | | Sentiment | yes | yes | yes | | Speaker diarization | yes | yes | yes | | Translation | yes | no | no | A session that asks a model for a feature it does not run (translation on `solaria-1`, for example) finds no model to serve it: `POST /v2/live` answers `500` with `no inference endpoint available`. Check the table before you combine a model and a feature. ## Latency profiles Each model trades latency for accuracy with a **latency profile**: `fast`, `medium`, `balanced` or `precise`. Add it to the model's name, or set it alone with `profile`: ```json { "model": "solaria-3-fast" } ``` ```json { "model": "trantoria-1", "profile": "precise" } ``` `fast` gives the earliest partial and final transcripts, `precise` the most accurate ones. Your account may have a default profile; a session that names one uses its own. ## Choosing - **Voice agent**: `trantoria-1-fast` or `solaria-3-fast`, with [end-of-turn detection](/live/end-of-turn/). - **Meeting notes, call analytics**: `solaria-3` or `solaria-1` with `balanced` or `precise`, and [diarization](/features/speaker-diarization/) or one channel per speaker. - **Live captions in several languages**: `solaria-3` with [translation](/features/translation/). --- # Quickstart You will transcribe a WAV file as if it were spoken live: the same code transcribes a microphone or a phone call. ## Start with your coding agent Building with an AI coding agent (Claude Code, Cursor, Copilot…)? Paste this prompt, then describe what you build. It gives the agent the documentation and the rules that avoid the usual integration bugs. ```text title="Prompt for your coding agent" You are integrating the Gladia Enterprise real-time speech-to-text API. Documentation (read it before writing code, every page also exists in Markdown): - Everything in one file: https://eu1.docs.gladia.io/llms-full.txt - Index of pages: https://eu1.docs.gladia.io/llms.txt - OpenAPI 3.1 contract, WebSocket messages under x-websocket: https://eu1.docs.gladia.io/openapi.json How the API works: - Create a session with POST {API_URL}/v2/live, header x-gladia-key: . The body describes the audio (encoding, sample_rate, bit_depth, channels), the model and the options. The answer has an id and a WebSocket url. - Stream the audio to that url (binary frames of raw samples, about 100 ms each), read the JSON messages, send {"type": "stop_recording"} when done. The server sends the last results, then closes with code 1000. - API_URL is the account's region: https://eu1.api.gladia.io (Europe) or https://us1.api.gladia.io (United States). - Models: solaria-3 (multilingual, translation), solaria-1 (99 languages), trantoria-1 (lowest latency, set the language). Check the docs' features-by-model table before combining a model and a feature. Rules: 1. Use the official SDK: Python gladiaio-sdk, JavaScript @gladiaio/sdk, with api_url / apiUrl set to API_URL. It buffers the audio until acknowledged and reconnects without losing words. Do not hand-write the WebSocket client unless the SDK cannot do the job. 2. Python SDK: LiveV2InitRequest silently drops the Gladia Enterprise options (numerals, pii, end_of_turn, diarization, punctuation, profile, turn_router, named_entity_recognition with options). For those, create the session with an HTTP POST, then call client.live_v2().connect_session(LiveV2ConnectSessionOptions(id, url)). The Python SDK reports pii and end_of_turn messages as errors: read them with the JavaScript SDK or a raw WebSocket. The JavaScript SDK sends any option and delivers every message. 3. Never ship the API key to a browser or a mobile app: create the session on the server, give the client only the session url (single use, valid 60 seconds). 4. The audio format in the init body must match the audio exactly; send raw samples, no WAV header, channels interleaved, little-endian. 5. Partial transcripts replace each other: key them by data.id and keep only the final (is_final: true). 6. Keep the API key in an environment variable (GLADIA_API_KEY), never in code. 7. Handle failures: POST /v2/live errors have a JSON body with message and validation_errors; a WebSocket closed with 1011 carries the reason (rejected: ... means an option the model refused); 429 means too many open sessions; 503 means retry with a backoff; 500 "no inference endpoint available" means no model in the region serves that combination. 8. Each feature page of the docs has a "Limits and gotchas" section: read the one of every feature you use. 9. Test with a real 16-bit WAV file end to end before wiring a microphone or a phone line. ``` Or follow the steps yourself: ## 1. Get an API key Sign in to **Gladia Workspace**, open **API keys** and create a key. A key restricted to some products needs the **Real-time STT** scope. Keep it on your server: anyone with the key can use your account. ```bash export GLADIA_API_KEY="ge_..." ``` ## 2. Install the SDK > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients. ```bash pip install gladiaio-sdk # Python npm install @gladiaio/sdk # JavaScript and TypeScript ``` ## 3. Transcribe a file Save the program, then run it on a 16-bit PCM WAV file: `python transcribe.py meeting.wav` or `node transcribe.mjs meeting.wav`. It prints each sentence once it is final. **Python SDK** ```python title="transcribe.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="transcribe.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **Python, no SDK** The same session with a raw WebSocket. You then handle acknowledgments and reconnections yourself ([Reconnections](/live/reconnections/)). ```python title="transcribe_raw.py" # pip install requests websockets import asyncio import json import os import sys import wave import requests import websockets API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") async def main(path: str) -> None: with wave.open(path) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) created = requests.post( f"{API_URL}/v2/live", headers={"x-gladia-key": os.environ["GLADIA_API_KEY"]}, json={"encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels}, timeout=10, ) created.raise_for_status() async with websockets.connect(created.json()["url"]) as ws: async def send() -> None: step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): await ws.send(audio[start : start + step]) # live audio: send each chunk as it is captured await ws.send(json.dumps({"type": "stop_recording"})) sender = asyncio.create_task(send()) async for raw in ws: # the server closes the connection once everything is sent message = json.loads(raw) if message["type"] == "transcript" and message["data"]["is_final"]: print(message["data"]["utterance"]["text"]) await sender asyncio.run(main(sys.argv[1])) ``` ``` Hello, thanks for calling. How can I help you? I'd like to change my delivery address. ``` ## What happened 1. The program **created a session** with `POST /v2/live`: its audio format, model and options. The answer has the session `id` and a WebSocket `url`, valid 60 seconds and for this session only: ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{"encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3"}' ``` ```json { "id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "created_at": "2026-10-12T09:04:17.210Z", "url": "wss://eu1.api.gladia.io/v2/live?token=eyJhbGciOi..." } ``` 2. It **streamed the audio** to that URL in 100 ms chunks. 3. It **printed the final transcripts** as they came, then stopped the recording: the API sent the last ones and closed the connection. The full result stays available with [`GET /v2/live/{id}`](/api-reference/v2/live/get/). ## Common mistakes - **The audio format does not match the configuration.** `sample_rate`, `bit_depth` and `channels` must describe what you send: wrong values give gibberish or nothing, never an error. The programs above read them from the file. - **A WAV header in a raw stream.** Send raw samples, without the 44-byte header, when you stream from a microphone or a phone line. - **The API key in a browser or a mobile app.** Create the session on your server and give the client only its `url` ([Authentication](/introduction/authentication/)). - **The URL used too late.** It expires 60 seconds after `POST /v2/live`: create the session when the audio is about to start. - **Wrong region.** Your key works on your account's region only ([Regions](/introduction/regions/)): a key of `eu1` gets a `401` on `us1`. ## Next - [Models](/introduction/models/): choose speed or accuracy. - [How a session works](/live/how-it-works/): the messages and their order. - [Audio intelligence](/features/translation/): translation, entities, PII redaction, sentiment. --- # Regions The API is served in several regions. Each region processes and stores its sessions on its own: audio and transcripts never leave the region of the address you call. | region | API | WebSocket | Gladia Workspace | |---|---|---|---| | Europe | `https://eu1.api.gladia.io` | `wss://eu1.api.gladia.io` | `https://eu1.app.gladia.io` | | United States | `https://us1.api.gladia.io` | `wss://us1.api.gladia.io` | `https://us1.app.gladia.io` | Your account, its API keys and its history belong to one region: use that region's addresses, shown in Gladia Workspace. The `region` parameter of `POST /v2/live` (`eu`, `us`) is accepted for compatibility with Gladia's cloud API. A region the address does not serve finds no model, and the session is refused. --- # SDKs The official Gladia SDKs work with the Gladia Enterprise API: set the API address of your region. > **Use the SDK.** The SDKs create the session, stream the audio, keep each chunk until the API has transcribed it, and reconnect after a network drop, resending what was not transcribed. That is the hard part of a live integration: most support requests we receive come from hand-written WebSocket clients. ```bash pip install gladiaio-sdk # Python 3.10 or later npm install @gladiaio/sdk # Node.js 20 or later, and browsers ``` ## Options of Gladia's cloud API Language, endpointing, custom vocabulary and spelling, translation, sentiment, callbacks, `custom_metadata`: pass them to `start_session` / `startSession` as usual. **Python SDK** ```python title="sdk_session.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "realtime_processing": { "custom_vocabulary": True, "custom_vocabulary_config": { "vocabulary": [ "Gladia" ] } } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="sdk_session.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, realtime_processing: { custom_vocabulary: true, custom_vocabulary_config: { vocabulary: [ "Gladia" ] } } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` `GLADIA_API_KEY` and `GLADIA_API_URL` in the environment work too. ## Options of Gladia Enterprise `profile`, `punctuation`, `numerals`, `end_of_turn`, `diarization`, `named_entity_recognition` with options, `pii` and `turn_router` are not in the SDKs' types yet (gladiaio/sdk#61). - **JavaScript**: pass them anyway, the SDK sends the configuration as is. In TypeScript, the configuration needs a cast (`as LiveV2InitRequest`). - **Python**: `LiveV2InitRequest` **drops** the fields it does not know, without an error. Create the session over HTTP, then hand it to the SDK with `connect_session`: it streams, buffers and reconnects as usual. **Python SDK** ```python title="sdk_enterprise_options.py" # pip install gladiaio-sdk requests import os import sys import threading import wave import requests from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "numerals": True, "end_of_turn": True } client = GladiaClient(api_key=API_KEY, api_url=API_URL) # `numerals` and `end_of_turn` are Gladia Enterprise options the Python SDK does not send yet # (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it. created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10) created.raise_for_status() info = created.json() session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"])) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) def on_error(error): if "Unsupported websocket message type" not in str(error): # end_of_turn messages: gladiaio/sdk#61 errors.append(error) session.on("error", on_error) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="sdk_enterprise_options.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, numerals: true, end_of_turn: true }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `numerals` and `end_of_turn` are Gladia Enterprise options not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text); if (message.type === "end_of_turn") console.log(" (end of turn)"); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` ## Limits and gotchas - **Messages the Python SDK does not know** (`pii`, `end_of_turn`, `end_of_turn_score`, `error`) reach your `error` handler as "Unsupported websocket message type", and are lost to it. Read them with the JavaScript SDK or a raw WebSocket ([Live WebSocket](/api-reference/v2/live/websocket/)). The JavaScript SDK delivers every message. - **Fields the Python SDK does not know on known messages** (`text_raw` and `entities` of an utterance, the `speaker` of a word) are not in its objects. The JavaScript SDK keeps them. - **`model`** is typed `"solaria-1"` in both SDKs: `solaria-3` and `trantoria-1` work, with a cast in TypeScript. - **In a browser**, never put the API key in the page: create the session on your server and give the page its `url`, then use `connectSession` ([Authentication](/introduction/authentication/#keep-the-key-on-your-server)). - **`region`**: leave the SDKs' `region` option unset; the API address you set is the region. --- # Code switching With **code switching**, the language is detected as the conversation goes: a call that moves from English to Spanish and back is transcribed in the language actually spoken. ```json { "model": "solaria-3", "language_config": { "languages": ["en", "es"], "code_switching": true } } ``` - List the languages that may be spoken: detection among a few languages is more reliable than among all of them. Without `code_switching`, only the first language listed is used. - An empty list allows every language of the model. - Each utterance carries the `language` it was transcribed in; `result.transcription.languages` lists them all, in order of appearance. Code switching works with `solaria-3` and `solaria-1`. --- # Language detection `language_config` tells the API which language to transcribe: | `language_config` | effect | |---|---| | `{}`, or `languages: []` | the language is detected | | `{"languages": ["fr"]}` | the session is transcribed in French | | `{"languages": ["en", "fr"]}` | the first language only, English | | `{"languages": ["en", "fr"], "code_switching": true}` | the language is detected, among English and French | ```json { "model": "solaria-3", "language_config": { "languages": ["fr"] } } ``` Every utterance carries its `language`, and the session's result lists the languages heard (`result.transcription.languages`). Setting the language when you know it gives faster and more accurate transcripts than detecting it, especially at the start of a session. Detection works with `solaria-3` and `solaria-1`. `trantoria-1` needs the language set ([Supported languages](/languages/supported-languages/)). ## Several languages in one session When speakers may use several languages, list them and turn on [code switching](/languages/code-switching/). --- # Supported languages Set languages with their ISO 639-1 codes in `language_config.languages`. A code the API does not know is refused with a `400` that lists it. | language | code | models | |---|---|---| | Afrikaans | `af` | solaria-1 | | Albanian | `sq` | solaria-1 | | Amharic | `am` | solaria-1 | | Arabic | `ar` | solaria-3, solaria-1 | | Armenian | `hy` | solaria-1 | | Assamese | `as` | solaria-1 | | Azerbaijani | `az` | solaria-1 | | Bangla | `bn` | solaria-1 | | Bashkir | `ba` | solaria-1 | | Basque | `eu` | solaria-1 | | Belarusian | `be` | solaria-1 | | Bosnian | `bs` | solaria-1 | | Breton | `br` | solaria-1 | | Bulgarian | `bg` | solaria-1 | | Burmese | `my` | solaria-1 | | Catalan | `ca` | solaria-1 | | Chinese | `zh` | solaria-3, solaria-1 | | Croatian | `hr` | solaria-1 | | Czech | `cs` | solaria-3, solaria-1 | | Danish | `da` | solaria-3, solaria-1 | | Dutch | `nl` | solaria-3, solaria-1 | | English | `en` | solaria-3, solaria-1 | | Estonian | `et` | solaria-1 | | Faroese | `fo` | solaria-1 | | Filipino | `tl` | solaria-3, solaria-1 | | Finnish | `fi` | solaria-3, solaria-1 | | French | `fr` | solaria-3, solaria-1 | | Galician | `gl` | solaria-1 | | Georgian | `ka` | solaria-1 | | German | `de` | solaria-3, solaria-1 | | Greek | `el` | solaria-3, solaria-1 | | Gujarati | `gu` | solaria-1 | | Haitian Creole | `ht` | solaria-1 | | Hausa | `ha` | solaria-1 | | Hawaiian | `haw` | solaria-1 | | Hebrew | `he` | solaria-1 | | Hindi | `hi` | solaria-3, solaria-1 | | Hungarian | `hu` | solaria-3, solaria-1 | | Icelandic | `is` | solaria-1 | | Indonesian | `id` | solaria-3, solaria-1 | | Italian | `it` | solaria-3, solaria-1 | | Japanese | `ja` | solaria-3, solaria-1 | | Javanese | `jw` | solaria-1 | | Kannada | `kn` | solaria-1 | | Kazakh | `kk` | solaria-1 | | Khmer | `km` | solaria-1 | | Korean | `ko` | solaria-3, solaria-1 | | Lao | `lo` | solaria-1 | | Latin | `la` | solaria-1 | | Latvian | `lv` | solaria-1 | | Lingala | `ln` | solaria-1 | | Lithuanian | `lt` | solaria-1 | | Luxembourgish | `lb` | solaria-1 | | Macedonian | `mk` | solaria-3, solaria-1 | | Malagasy | `mg` | solaria-1 | | Malay | `ms` | solaria-3, solaria-1 | | Malayalam | `ml` | solaria-1 | | Maltese | `mt` | solaria-1 | | Māori | `mi` | solaria-1 | | Marathi | `mr` | solaria-1 | | Mongolian | `mn` | solaria-1 | | Nepali | `ne` | solaria-1 | | Norwegian | `no` | solaria-1 | | Norwegian Nynorsk | `nn` | solaria-1 | | Occitan | `oc` | solaria-1 | | Pashto | `ps` | solaria-1 | | Persian | `fa` | solaria-3, solaria-1 | | Polish | `pl` | solaria-3, solaria-1 | | Portuguese | `pt` | solaria-3, solaria-1 | | Punjabi | `pa` | solaria-1 | | Romanian | `ro` | solaria-3, solaria-1 | | Russian | `ru` | solaria-3, solaria-1 | | Sanskrit | `sa` | solaria-1 | | Serbian | `sr` | solaria-1 | | Shona | `sn` | solaria-1 | | Sindhi | `sd` | solaria-1 | | Sinhala | `si` | solaria-1 | | Slovak | `sk` | solaria-1 | | Slovenian | `sl` | solaria-1 | | Somali | `so` | solaria-1 | | Spanish | `es` | solaria-3, solaria-1 | | Sundanese | `su` | solaria-1 | | Swahili | `sw` | solaria-1 | | Swedish | `sv` | solaria-3, solaria-1 | | Tajik | `tg` | solaria-1 | | Tamil | `ta` | solaria-1 | | Tatar | `tt` | solaria-1 | | Telugu | `te` | solaria-1 | | Thai | `th` | solaria-3, solaria-1 | | Tibetan | `bo` | solaria-1 | | Turkish | `tr` | solaria-3, solaria-1 | | Turkmen | `tk` | solaria-1 | | Ukrainian | `uk` | solaria-1 | | Urdu | `ur` | solaria-1 | | Uzbek | `uz` | solaria-1 | | Vietnamese | `vi` | solaria-3, solaria-1 | | Welsh | `cy` | solaria-1 | | Yiddish | `yi` | solaria-1 | | Yoruba | `yo` | solaria-1 | Translation targets (solaria-3): Afrikaans (`af`), Amharic (`am`), Arabic (`ar`), Assamese (`as`), Belarusian (`be`), Bulgarian (`bg`), Bangla (`bn`), Catalan (`ca`), Danish (`da`), German (`de`), English (`en`), Spanish (`es`), Persian (`fa`), French (`fr`), Galician (`gl`), Indonesian (`id`), Italian (`it`), Japanese (`ja`), Khmer (`km`), Korean (`ko`), Lingala (`ln`), Lao (`lo`), Malay (`ms`), Burmese (`my`), Nepali (`ne`), Dutch (`nl`), Norwegian (`no`), Pashto (`ps`), Portuguese (`pt`), Romanian (`ro`), Russian (`ru`), Somali (`so`), Swedish (`sv`), Thai (`th`), Vietnamese (`vi`), Wolof (`wo`), Yoruba (`yo`), Chinese (`zh`). **trantoria-1** transcribes about 40 languages, English and Spanish included; it does not detect the language: set the one spoken. A session asking a model for a language it does not transcribe is refused when it starts. For translation targets, see [Translation](/features/translation/#target-languages). --- # Limits | limit | value | |---|---| | Concurrent sessions | set by your plan | | Session duration | 8 hours by default; your plan may set another | | Connecting to a session's URL | within 60 seconds of `POST /v2/live` | | Silence before a session stops | 60 seconds without audio | | Reconnecting after a drop | within 60 seconds | | `GET /v2/live` page size | 100 sessions | ## Concurrent sessions Your account may have a number of sessions open at once. Past it, `POST /v2/live` answers `429`, and so does a WebSocket connection to a session created earlier. A session counts from its creation until it ends, or until its URL expires unused. End sessions with `stop_recording` rather than letting them time out. ## Session duration A session that reaches its maximum duration stops as if you had sent `stop_recording`: its last transcripts and results arrive as usual. For longer streams, start a new session before the limit and switch to it. ## Retrying | answer | what to do | |---|---| | `429` | Wait for one of your sessions to end, or ask for a higher limit. | | `503` | The service is busy: retry after a short delay, with backoff. | | `500` "no inference endpoint available" | The model you asked for has no capacity in this region right now: retry later, or use another model. | --- # Audio formats Describe the audio you send in the init body. The API reads your stream with these values: they must match it. | field | values | default | |---|---|---| | `encoding` | `wav/pcm` (linear PCM), `wav/alaw`, `wav/ulaw` (G.711, telephony) | `wav/pcm` | | `bit_depth` | `8`, `16`, `24`, `32`; `wav/alaw` and `wav/ulaw` are 8-bit | `16` | | `sample_rate` | `8000`, `16000`, `32000`, `44100`, `48000` | `16000` | | `channels` | `1` to `8` | `1` | Send the samples raw, without a WAV header: little-endian, channels interleaved. ```json { "encoding": "wav/ulaw", "sample_rate": 8000, "bit_depth": 8, "channels": 2 } ``` ## Sending the audio - **Binary frames** (recommended): the bytes as they are. - **JSON**: `{"type": "audio_chunk", "data": {"chunk": ""}}`, a third larger. Any chunk size works. 100 ms per chunk is a good default: smaller chunks add overhead, larger ones delay the transcripts. Send the audio as it is captured. ## Several channels Each channel is transcribed on its own: utterances carry their `channel` (from `0`), and utterance ids start with it (`01_00000004`). Put each party of a call on its own channel to know who said what without [diarization](/features/speaker-diarization/), and with exact timing when they talk over each other. ## Acknowledgments With `receive_acknowledgments` (on by default), the server acknowledges your audio once it is transcribed, with `audio_chunk` messages whose `byte_range` counts the bytes since the start of the session. Keep the audio until it is acknowledged to resend it after a network drop ([Reconnections](/live/reconnections/)). --- # Callbacks With callbacks, the API also POSTs a session's messages to your server: to store transcripts as they come, or process them away from the client that streams the audio. ```json { "callback": true, "callback_config": { "url": "https://hooks.example.com/gladia", "receive_final_transcripts": true, "receive_post_processing_events": true, "receive_lifecycle_events": true } } ``` `callback_config` chooses the messages independently of the WebSocket's `messages_config`; its defaults differ (final transcripts, post-processing and lifecycle events on, the rest off). The URL must be `https`, on a public address. ## Payload Each POST carries the session id, the event and the WebSocket message: ```json { "id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "event": "live.transcript", "payload": { "type": "transcript", "session_id": "0199d7a2-...", "data": { "id": "00_00000003", "is_final": true, "utterance": { "text": "..." } } } } ``` `event` is `live.` followed by the message type: `live.start_session` (callbacks only, when the WebSocket opens), `live.start_recording`, `live.transcript`, `live.speech_start`, `live.speech_end`, `live.translation`, `live.named_entity_recognition`, `live.sentiment_analysis`, `live.audio_chunk`, `live.stop_recording`, `live.end_recording`, `live.post_transcript`, `live.post_final_transcript`, `live.end_session`, `live.error`. The payloads are the [WebSocket messages](/api-reference/v2/live/websocket/#receive). `pii` and `end_of_turn` messages are not sent to callbacks. ## Verify the signature Each POST is signed with your workspace's callback signing secret, in the header: ``` x-gladia-signature: t=,v1= ``` `v1` is the hex HMAC-SHA256, keyed with the secret, of `.` followed by the **raw** request body (the bytes as received, before any JSON parsing). To verify a callback: 1. Split the header on `,` and read `t` and `v1`. 2. Compute `HMAC-SHA256(secret, "." + raw body)` and compare it with `v1` in constant time. 3. Refuse a `t` older than your replay window (5 minutes is a good default). ```python import hashlib, hmac, time def verify(secret: str, header: str, raw_body: bytes, window_s: int = 300) -> bool: parts = dict(p.split("=", 1) for p in header.split(",")) t, v1 = parts.get("t", ""), parts.get("v1", "") if not t.isdigit() or abs(time.time() - int(t)) > window_s: return False expected = hmac.new(secret.encode(), f"{t}.".encode() + raw_body, hashlib.sha256).hexdigest() return hmac.compare_digest(expected, v1) ``` ```js import { createHmac, timingSafeEqual } from "node:crypto"; export function verify(secret, header, rawBody, windowS = 300) { const parts = Object.fromEntries(header.split(",").map((p) => p.split("=", 2))); if (!/^\d+$/.test(parts.t ?? "") || Math.abs(Date.now() / 1000 - Number(parts.t)) > windowS) return false; const expected = createHmac("sha256", secret).update(`${parts.t}.`).update(rawBody).digest("hex"); return expected.length === parts.v1?.length && timingSafeEqual(Buffer.from(expected), Buffer.from(parts.v1)); } ``` The secret (`whsec_…`) is created at your workspace's first callback. Owners and admins rotate it in Gladia Workspace, **Settings → Security → Callback signing secret**: the new secret is shown once, and the previous one stops signing at once. ## Delivery - Messages are sent in order, one at a time per connection; a slow or failing receiver never slows the session. - Network errors and `5xx` answers are retried, 3 attempts in all with a growing delay; `4xx` answers are not. - Each POST times out after 5 seconds. - After a session ends, its remaining messages are delivered for up to 30 seconds. - Up to 1,024 messages wait per connection; beyond, messages are dropped. - After a [reconnection](/live/reconnections/), messages of the old and new connections may interleave: order them by the utterance `id`. Answer `2xx` quickly and process the message afterwards. --- # End of turn A voice agent must answer when the caller has **finished**, not at every pause: "my number is… zero six…" has pauses, and is not finished. **End-of-turn detection** listens to how the speaker talks at each pause and sends an `end_of_turn` message when their turn is over. Turn it on in the init body: ```json { "end_of_turn": true } ``` Then answer when an `end_of_turn` message arrives. Each program prints the caller's sentences and each end of turn: **JavaScript SDK** ```js title="end_of_turn.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "trantoria-1", language_config: { languages: [ "en" ] }, endpointing: 0.3, end_of_turn: { threshold: 0.8, stop_s: 1.2 }, messages_config: { receive_partial_transcripts: true } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); // `end_of_turn` is a Gladia Enterprise option not in the SDK's types yet: the SDK sends it as is. const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) console.log("caller:", message.data.utterance.text); if (message.type === "end_of_turn") console.log(`end of turn (${message.data.reason}): answer now`); }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **Python** ```python title="end_of_turn.py" # pip install requests websockets # The Python SDK does not read end_of_turn messages yet (gladiaio/sdk#61): a raw WebSocket here. import asyncio import json import os import sys import wave import requests import websockets API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") async def main(path: str) -> None: with wave.open(path) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) created = requests.post( f"{API_URL}/v2/live", headers={"x-gladia-key": os.environ["GLADIA_API_KEY"]}, json={ "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "trantoria-1", "language_config": {"languages": ["en"]}, "endpointing": 0.3, "end_of_turn": {"threshold": 0.8, "stop_s": 1.2}, }, timeout=10, ) created.raise_for_status() async with websockets.connect(created.json()["url"]) as ws: async def send() -> None: step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): await ws.send(audio[start : start + step]) # live audio: send each chunk as it is captured await ws.send(json.dumps({"type": "stop_recording"})) sender = asyncio.create_task(send()) async for raw in ws: message = json.loads(raw) if message["type"] == "transcript" and message["data"]["is_final"]: print("caller:", message["data"]["utterance"]["text"]) elif message["type"] == "end_of_turn": print(f"end of turn ({message['data']['reason']}): answer now") await sender asyncio.run(main(sys.argv[1])) ``` > **Python SDK.** The Python SDK does not read `end_of_turn` messages yet: they reach its `error` handler and are lost (gladiaio/sdk#61). Use the JavaScript SDK, or a raw WebSocket as above. The message: ```json { "type": "end_of_turn", "session_id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "data": { "channel": 0, "probability": 0.97, "reason": "model", "end": 18.42, "speech_duration": 3.1 } } ``` `reason` is `model` when the probability that the speaker finished crossed `threshold`, or `timeout` when the silence reached `stop_s` whatever the probability. ## Options `"end_of_turn": true` uses the defaults; an object turns it on and changes some of them. Omitted or `false`: off. ```json { "end_of_turn": { "threshold": 0.8, "stop_s": 1.2 } } ``` | option | range | what it does | |---|---|---| | `threshold` | 0 to 1 (excluded) | Probability above which the turn ends. Lower: faster answers, more interruptions. | | `silence_ms` | 1 to 10000 | Silence, in ms, before a pause is evaluated. | | `stop_s` | up to 60 | Silence, in seconds, that ends the turn anyway. | | `emit_scores` | | Also send `end_of_turn_score` for each evaluation, to calibrate `threshold`. | | `window_s` | up to 8 | Audio, in seconds, an evaluation reads. | | `speech_dbfs` | -100 to 0 | Level under which audio counts as silence. | | `reeval_ms` | 0 to 10000 | Interval between two evaluations of the same pause. | [`GET /v2/live/defaults`](/api-reference/v2/live/defaults/) returns the defaults in force. ## Building a voice agent 1. Use a low-latency model and profile: `trantoria-1-fast` or `solaria-3-fast`. 2. Turn on partial transcripts to prepare the answer while the caller speaks. 3. Answer on `end_of_turn`, with the finals received so far. 4. If the caller speaks again before you answer, cancel and wait for the next `end_of_turn`. ## Limits and gotchas - **Off unless you ask.** A session without `end_of_turn` gets no end-of-turn message. - **WebSocket only.** `end_of_turn` messages are never sent to callbacks: the agent that answers must hold the WebSocket. - **Per channel.** Each channel has its own turns: with the caller and the agent on two channels, read `channel`. - **`threshold` is a trade-off.** Lower answers sooner and interrupts more; higher waits for clearer endings. Turn on `emit_scores` on real calls to see the probabilities, then set it. - **`stop_s` is your ceiling.** Whatever the probability, a silence that long ends the turn (`reason: timeout`): keep it under what your callers will tolerate. - **Endpointing still applies.** Finals still come on each pause: buffer them, answer on `end_of_turn`. - **Python SDK**: see above, use a raw WebSocket or the JavaScript SDK. --- # Endpointing **Endpointing** decides when an utterance ends: after a pause of `endpointing` seconds, its final transcript is sent. It sets the trade-off between fast finals and whole sentences. | field | range | when omitted | |---|---|---| | `endpointing` | 0.01 to 10 s | 0.64 s | | `maximum_duration_without_endpointing` | 5 to 60 s | no limit | ```json { "endpointing": 0.3, "maximum_duration_without_endpointing": 15 } ``` In a session, with the SDKs: **Python SDK** ```python title="endpointing.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "endpointing": 0.3, "maximum_duration_without_endpointing": 15 } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript" and message.data.is_final: print(message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="endpointing.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, endpointing: 0.3, maximum_duration_without_endpointing: 15 }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript" && message.data.is_final) { const { text, end, forced } = message.data.utterance; console.log(`${end.toFixed(1)}s ${forced ? "(forced) " : ""}${text}`); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "endpointing": 0.3, "maximum_duration_without_endpointing": 15 }' ``` - A **short** `endpointing` (0.2 to 0.4 s) gives faster finals, and cuts sentences at short hesitations. - A **long** one (0.8 s and more) gives whole sentences, later. - `maximum_duration_without_endpointing` forces a final after that much speech without a pause; such an utterance carries `"forced": "max_duration"`. [`GET /v2/live/defaults`](/api-reference/v2/live/defaults/) returns the values in force for your account. For a voice agent, keep endpointing for the transcript and use [end-of-turn detection](/live/end-of-turn/) to decide when to answer: a pause in the middle of a sentence is not the end of a turn. ## Limits and gotchas - **Too short cuts words.** Under 0.2 s, a breath or a hesitation ends the utterance: finals split mid-sentence. - **`maximum_duration_without_endpointing` cuts anywhere.** A forced final can end mid-word: keep it for monologues and noisy lines without pauses. - **It is not end of turn.** A pause ends an utterance, not the speaker's turn: voice agents should answer on [end of turn](/live/end-of-turn/). - **Python SDK**: an utterance's `forced` is not in the SDK's objects (gladiaio/sdk#61); the JavaScript SDK has it. ## Number hold A phone number, an amount or an address dictated in pieces ("my card number is 4970… 1234… 5678") would be cut into several finals, one per pause. The **number hold** keeps the utterance open while the number is still being dictated, up to 2 seconds without new words, then closes it as usual. It is **on by default** where the model supports it ([Models](/introduction/models/#features-by-model)); the session is never refused for it. Turn it off with: ```json { "turn_router": false } ``` --- # How a session works A live session is one stream of audio and everything transcribed from it. ## Lifecycle 1. **Create** the session: [`POST /v2/live`](/api-reference/v2/live/init/) with its options. It answers the session `id` and a WebSocket `url`, valid 60 seconds. 2. **Connect** to the `url`. With `messages_config.receive_lifecycle_events`, a [`start_recording`](/api-reference/v2/live/message/start-recording/) message says the session is ready. 3. **Stream** the audio, as binary frames or [`audio_chunk`](/api-reference/v2/live/websocket/#send) messages. Transcripts arrive as the audio is processed. 4. **Stop** with `{"type": "stop_recording"}`. The server transcribes what is left, sends `end_recording`, the post-processing results ([`post_transcript`](/api-reference/v2/live/message/post-transcript/), [`post_final_transcript`](/api-reference/v2/live/message/post-final-transcript/)), `end_session`, then closes the connection with code `1000`. A session also stops after 60 seconds without audio, and when it reaches its maximum duration ([Limits](/limits/concurrency/)). ## Utterances The audio is cut into **utterances** at pauses ([Endpointing](/live/endpointing/)). Each utterance has an `id`, `_` (`00_00000003`), stable for the whole session. While it is spoken you receive **partial** transcripts (`is_final: false`, with `receive_partial_transcripts`), each replacing the previous one; when it ends, one **final** transcript (`is_final: true`). ```json { "type": "transcript", "session_id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "created_at": "2026-10-12T09:04:19.000Z", "data": { "id": "00_00000003", "is_final": true, "utterance": { "text": "I'd like to change my delivery address.", "start": 12.48, "end": 14.9, "language": "en", "confidence": 0.94, "channel": 0, "words": [{ "word": "I'd", "start": 12.48, "end": 12.62, "confidence": 0.97 }] } } } ``` The features you turn on add their own messages right after each final (translation, entities, redacted data, sentiment), named by the same utterance `id`. ## Choosing your messages `messages_config` in the init body selects what comes on the WebSocket: | option | messages | default | |---|---|---| | `receive_partial_transcripts` | `transcript` with `is_final: false` | `false` | | `receive_final_transcripts` | `transcript` with `is_final: true` | `true` | | `receive_speech_events` | `speech_start`, `speech_end` | `true` | | `receive_realtime_processing_events` | `translation`, `named_entity_recognition`, `sentiment_analysis` | `true` | | `receive_post_processing_events` | `post_transcript`, `post_final_transcript` | `true` | | `receive_acknowledgments` | `audio_chunk`, `stop_recording` acknowledgments | `true` | | `receive_lifecycle_events` | `start_recording`, `end_recording`, `end_session` | `false` | | `receive_errors` | `error` | `true` | `pii` and `end_of_turn` messages come whenever their feature is on. Every message type is described in the [API reference](/api-reference/v2/live/websocket/#receive). The same messages can also go to your server ([Callbacks](/live/callbacks/)). ## After the session [`GET /v2/live/{id}`](/api-reference/v2/live/get/) returns the session's options and, once it is `done`, its result: every final utterance, the full transcript and the translations. Sessions stay in your history for your account's retention ([Data retention](/data/retention/)). --- # Partial transcripts A **partial** transcript is the text of the utterance being spoken, so far. It changes as more is heard, until the utterance ends and its **final** transcript arrives. Partials make live captions feel instant and let a voice agent start thinking before the speaker pauses. > **Use the SDK.** The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients. They are off by default. Turn them on in the init body: ```json { "messages_config": { "receive_partial_transcripts": true } } ``` Partials and the final of an utterance share its `data.id`: replace the displayed text of that id with each new message, and keep the final. Each program prints both: **Python SDK** ```python title="partials.py" # pip install gladiaio-sdk import os import sys import threading import wave from gladiaio_sdk import GladiaClient, LiveV2InitRequest API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io") API_KEY = os.environ["GLADIA_API_KEY"] with wave.open(sys.argv[1]) as wav: # a 16-bit PCM WAV file rate, channels = wav.getframerate(), wav.getnchannels() audio = wav.readframes(wav.getnframes()) config = { "encoding": "wav/pcm", "sample_rate": rate, "bit_depth": 16, "channels": channels, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "messages_config": { "receive_partial_transcripts": True } } client = GladiaClient(api_key=API_KEY, api_url=API_URL) session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config)) ended, errors = threading.Event(), [] @session.on("message") def on_message(message): if message.type == "transcript": kind = "final " if message.data.is_final else "partial" print(kind, message.data.id, message.data.utterance.text) session.on("error", errors.append) session.once("ended", lambda _: ended.set()) step = rate * channels * 2 // 10 # 100 ms of audio per chunk for start in range(0, len(audio), step): session.send_audio(audio[start : start + step]) # live audio: send each chunk as it is captured session.stop_recording() ended.wait(timeout=60) sys.exit(1 if errors or not ended.is_set() else 0) ``` **JavaScript SDK** ```js title="partials.mjs" // npm install @gladiaio/sdk import { readFileSync } from "node:fs"; import { GladiaClient } from "@gladiaio/sdk"; const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file const config = { encoding: "wav/pcm", sample_rate: sampleRate, bit_depth: 16, channels, model: "solaria-3", language_config: { languages: [ "en" ] }, messages_config: { receive_partial_transcripts: true } }; const client = new GladiaClient({ apiKey: process.env.GLADIA_API_KEY, apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io", }); const session = client.liveV2().startSession(config); session.on("message", (message) => { if (message.type === "transcript") { const kind = message.data.is_final ? "final " : "partial"; console.log(kind, message.data.id, message.data.utterance.text); } }); session.on("error", (error) => { console.error(error); process.exitCode = 1; }); session.once("ended", () => process.exit()); const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk for (let start = 0; start < pcm.length; start += step) { session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured } session.stopRecording(); function readWav(path) { const file = readFileSync(path); let fmt, pcm; for (let pos = 12; pos < file.length; ) { const id = file.toString("ascii", pos, pos + 4); const size = file.readUInt32LE(pos + 4); if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) }; if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size); pos += 8 + size + (size % 2); } return { ...fmt, pcm }; } ``` **HTTP** Create the session, then stream its audio to the returned `url` ([Live WebSocket](/api-reference/v2/live/websocket/)): ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "messages_config": { "receive_partial_transcripts": true } }' ``` ## Limits and gotchas - **Partials are not final**: words may change, and confidence is lower. Store and process the finals only. - **Do not append partials**: replace the text of their `id`. Appending them gives repeated words on screen. - **More messages, more traffic**: a partial comes several times a second per speaker. Turn them off (`receive_partial_transcripts: false`, the default) if you only store transcripts. - With [PII redaction](/features/pii-redaction/), no partial is sent: only the finals are redacted. - The [latency profile](/introduction/models/#latency-profiles) decides how often partials come: `fast` sends them earliest. - Partials can also go to your [callback](/live/callbacks/) (`callback_config.receive_partial_transcripts`). --- # Recommended parameters Starting points to adapt. Every option is described in [`POST /v2/live`](/api-reference/v2/live/init/). ## Voice agents Fast transcripts, and an answer only when the caller has finished. ```json { "model": "trantoria-1-fast", "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "language_config": { "languages": ["en"] }, "endpointing": 0.3, "end_of_turn": true, "numerals": true, "messages_config": { "receive_partial_transcripts": true } } ``` Answer on [`end_of_turn`](/live/end-of-turn/), not on each final. The [number hold](/live/endpointing/#number-hold) keeps a dictated card or phone number in one utterance. ## Call centers Telephone audio, agent and caller on separate channels, personal data redacted. ```json { "model": "solaria-3", "encoding": "wav/ulaw", "sample_rate": 8000, "bit_depth": 8, "channels": 2, "endpointing": 0.5, "pii": true, "numerals": true, "realtime_processing": { "sentiment_analysis": true }, "custom_metadata": { "call_id": "..." } } ``` ## Meetings One microphone, several speakers, whole sentences, your jargon. ```json { "model": "solaria-3-balanced", "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "endpointing": 0.8, "diarization": true, "named_entity_recognition": true, "realtime_processing": { "custom_vocabulary": true, "custom_vocabulary_config": { "vocabulary": ["Gladia", "solaria"] } } } ``` ## Live captions in several languages ```json { "model": "solaria-3-fast", "encoding": "wav/pcm", "sample_rate": 48000, "bit_depth": 16, "channels": 1, "language_config": { "languages": [], "code_switching": true }, "messages_config": { "receive_partial_transcripts": true }, "realtime_processing": { "translation": true, "translation_config": { "target_languages": ["en", "fr", "es"] } } } ``` --- # Reconnections A live session outlives its WebSocket. If the connection drops (your network, or a server of ours restarting), reconnect to **the same URL**: the session continues with the same id and the next utterance ids, without lost or repeated transcripts, and its final result covers everything. ## With the Gladia SDKs Nothing to do. The SDKs reconnect to the same URL when the connection closes abnormally, keep every audio chunk until it is acknowledged, and resend the unacknowledged ones. ## With your own client 1. Keep acknowledgments on (`messages_config.receive_acknowledgments`, the default). 2. Keep each audio chunk until an `audio_chunk` message covers it: `data.byte_range[1]` is the number of bytes acknowledged since the start of the session. Audio is acknowledged once it is transcribed. 3. When the connection closes with any code other than `1000`, reconnect to the same URL within 60 seconds, send the kept chunks first, then continue. Audio already transcribed is recognised and acknowledged again, never transcribed twice. 4. A `transcript` message in flight when the connection broke is not sent again; the session's result, its `post_final_transcript` and your callbacks have it. ## Close codes | code | meaning | what to do | |---|---|---| | `1000` | the session ended | nothing | | `1006` | the connection was lost | reconnect, resend the unacknowledged audio | | `1011` | the session failed | create a new session | | `1012` | a server is restarting | reconnect at once | | `4401` | the URL is invalid, expired, or the session ended | create a new session | While a dropped session may still resume, [`GET /v2/live/{id}`](/api-reference/v2/live/get/) reports it as `processing`; once the 60 seconds have passed, `done`. --- # From Gladia's cloud API The Gladia Enterprise API is Gladia's Live V2 API. To move an integration: 1. **Create an API key** in Gladia Workspace ([Authentication](/introduction/authentication/)). 2. **Change the base URL**: `api_url` / `apiUrl` in the [SDKs](/introduction/sdks/) (or `GLADIA_API_URL`), or the host of your HTTP calls: `https://eu1.api.gladia.io` instead of `https://api.gladia.io`. 3. Run your tests: the init body, the WebSocket messages, callbacks and the history are the same. ## What is the same `POST /v2/live` and its body, the WebSocket protocol and its messages, `messages_config`, callbacks and their payloads, `GET /v2/live`, `GET /v2/live/{id}`, `DELETE /v2/live/{id}`, `GET /v2/live/{id}/file`, error bodies. ## What differs | | Gladia Enterprise | |---|---| | Models | `solaria-3`, `solaria-1`, `trantoria-1`, with [latency profiles](/introduction/models/#latency-profiles) | | `endpointing` when omitted | 0.64 s ([Endpointing](/live/endpointing/)) | | `maximum_duration_without_endpointing` when omitted | no limit | | Acknowledgments | sent once the audio is transcribed, not on receipt: this makes [reconnections](/live/reconnections/) lossless | | Recordings | FLAC instead of WAV, for accounts that record | | History pagination | follow `next` (a `cursor`); `offset` up to 1,000, `limit` up to 100 | | `region` | the region is the API address you call ([Regions](/introduction/regions/)) | | Pre-recorded API (`/v2/pre-recorded`, `/v2/upload`) | not available | | Summarization, chapterization | not available: accepted, ignored | | `pre_processing` | accepted, not applied | ## Extensions Options that only the Gladia Enterprise API knows. Do not send them to Gladia's cloud API. | option | | |---|---| | `profile` | [latency profile](/introduction/models/#latency-profiles) | | `end_of_turn` | [end-of-turn detection](/live/end-of-turn/) | | `turn_router` | turns off the [number hold](/live/endpointing/#number-hold) | | `numerals` | [numbers as digits](/features/numerals/) | | `punctuation` | turns off [punctuation](/features/punctuation/) | | `diarization`, `diarization_config` | [speaker diarization](/features/speaker-diarization/) on live audio | | `named_entity_recognition` | [entities](/features/named-entity-recognition/), with options | | `pii` | [PII redaction](/features/pii-redaction/) | | `GET /v2/live/defaults` | the values of omitted options | --- # Playground A page of this site that runs a live session from the browser's microphone with your API key: POST /v2/live, then the WebSocket, 16-bit PCM at 16 kHz. It shows the partial and final transcripts, entities, redacted personal data, translations and sentiment, and the init body it sent. --- # Initiate a session `POST /v2/live` Creates a live session with its options and returns the WebSocket URL to stream its audio to. Authentication: your API key in the `x-gladia-key` header. ## Parameters - `region` (query, string, one of: `eu`, `us`, `eu-west`, `us-west`): Where the audio is processed. Every endpoint processes it in its own region (eu1: Europe, us1: United States); a region the endpoint does not serve finds no model and the session is refused. ## Body - `encoding` (string, default: `"wav/pcm"`, one of: `wav/pcm`, `wav/alaw`, `wav/ulaw`): Encoding of the audio you send. - `bit_depth` (integer, default: `16`, one of: `8`, `16`, `24`, `32`): Bit depth of the audio. `wav/alaw` and `wav/ulaw` are 8-bit. - `sample_rate` (integer, default: `16000`, one of: `8000`, `16000`, `32000`, `44100`, `48000`): Sample rate of the audio, in Hz. - `channels` (integer, default: `1`, 1 to 8): Number of interleaved audio channels. Each channel is transcribed on its own: put each speaker of a call on their own channel to know who said what without diarization. - `custom_metadata` (object): Any JSON object, kept with the session: returned by `GET /v2/live/{id}`, sent to your callback, and a filter of `GET /v2/live`. - `model` (string, one of: `solaria-3`, `solaria-1`, `trantoria-1`, `solaria-3-fast`, `solaria-3-medium`, `solaria-3-balanced`, `solaria-3-precise`, `solaria-1-fast`, `solaria-1-medium`, `solaria-1-balanced`, `solaria-1-precise`, `trantoria-1-fast`, `trantoria-1-medium`, `trantoria-1-balanced`, `trantoria-1-precise`, `fast`, `medium`, `balanced`, `precise`): The model, by name, optionally with a latency profile (`solaria-3-fast`), or a latency profile alone (`fast`). Omitted: the default model of your account. - `endpointing` (number, default: `0.64`, 0.01 to 10): Silence, in seconds, that ends an utterance and produces its final transcript. Short values give faster finals and may cut sentences at hesitations. - `maximum_duration_without_endpointing` (number, 5 to 60): Longest utterance, in seconds: a final is forced after this much speech without a pause. Omitted: no limit. - `language_config` (object): The languages to expect. - `languages` (array of string): The languages that may be spoken (ISO 639-1 codes). Empty: the language is detected. One or more codes: the session is transcribed in the first one, unless `code_switching`. - `code_switching` (boolean, default: `false`): Detect the language as the conversation goes, among `languages` when given: for speakers who switch languages. - `realtime_processing` (object): Processing applied to each final transcript, during the session. - `custom_vocabulary` (boolean, default: `false`): Bias the transcription toward the words of `custom_vocabulary_config`. - `custom_vocabulary_config` (object, default: `null`): The words to favour. Required with `custom_vocabulary`. - `vocabulary` (array of string or object, required): Words or phrases, as strings or as objects with their own options. - `value` (string, required): The word or phrase. - `intensity` (number, default: `null`, 0 to 1): How strongly it is favoured, from 0 to 1. - `pronunciations` (array of string, default: `null`): Other ways it may be pronounced, written as they sound. - `default_intensity` (number, default: `null`, 0 to 1): How strongly the words are favoured, from 0 to 1, when an entry sets none. - `custom_spelling` (boolean, default: `false`): Rewrite words as `custom_spelling_config` says. - `custom_spelling_config` (object, default: `null`): The spellings to apply. Required with `custom_spelling`. - `spelling_dictionary` (object, required): The spelling to write (key) for each variant heard (values): `{"Gladia": ["gladya", "glad ia"]}`. Whole words, case-insensitive. - `translation` (boolean, default: `false`): Translate each final transcript into the target languages. - `translation_config` (object, default: `null`): The target languages. Required with `translation`. - `target_languages` (array of string, required, at least 1 item(s)): The languages to translate into (ISO 639-1 codes). - `named_entity_recognition` (boolean, default: `false`): The same as the top-level `named_entity_recognition: true`. - `sentiment_analysis` (boolean, default: `false`): The sentiment (positive, neutral, negative) and emotion of each final, in a `sentiment_analysis` message after it. - `messages_config` (object): The messages sent on the WebSocket. - `receive_partial_transcripts` (boolean, default: `false`): Partial transcripts, updated while the utterance is spoken. - `receive_final_transcripts` (boolean, default: `true`): The final transcript of each utterance. - `receive_speech_events` (boolean, default: `true`): `speech_start` and `speech_end`. - `receive_pre_processing_events` (boolean, default: `true`): Pre-processing messages. - `receive_realtime_processing_events` (boolean, default: `true`): Messages of the realtime processing: `translation`, `named_entity_recognition`, `sentiment_analysis`. - `receive_post_processing_events` (boolean, default: `true`): `post_transcript` and `post_final_transcript`, at the end. - `receive_acknowledgments` (boolean, default: `true`): `audio_chunk` and `stop_recording` acknowledgments. - `receive_errors` (boolean, default: `true`): `error` messages. - `receive_lifecycle_events` (boolean, default: `false`): `start_recording`, `end_recording` and `end_session`. - `callback` (boolean, default: `false`): Also send the session's messages to `callback_config.url`, by HTTP POST. - `callback_config` (object, default: `null`): Where and which messages the callback receives. Required with `callback`. - `url` (string, required, uri): The HTTPS URL that receives the messages, one POST each. - `receive_partial_transcripts` (boolean, default: `false`): Partial transcripts. - `receive_final_transcripts` (boolean, default: `true`): Final transcripts. - `receive_speech_events` (boolean, default: `false`): `speech_start` and `speech_end`. - `receive_pre_processing_events` (boolean, default: `true`): Pre-processing messages. - `receive_realtime_processing_events` (boolean, default: `true`): Messages of the realtime processing. - `receive_post_processing_events` (boolean, default: `true`): `post_transcript` and `post_final_transcript`. - `receive_acknowledgments` (boolean, default: `false`): Acknowledgments. - `receive_errors` (boolean, default: `false`): `error` messages. - `receive_lifecycle_events` (boolean, default: `true`): `start_session`, `start_recording`, `end_recording` and `end_session`. - `profile` (string, default: `null`, one of: `fast`, `medium`, `balanced`, `precise`): Latency profile: `fast` gives the earliest transcripts, `precise` the most accurate ones. The same as a profile in `model`. - `diarization` (boolean, default: `false`): Speaker labels on single-channel audio: each utterance and word gets a `speaker` number. - `diarization_config` (object): Diarization options. - `number_of_speakers` (integer, at least 1): Expected number of speakers, when you know it. - `max_speakers` (integer, at least 1): Highest number of speakers to tell apart. - `punctuation` (boolean, default: `true`): Punctuation and casing of the transcripts. On unless `false`. - `end_of_turn` (boolean or object, default: `false`): End-of-turn detection for voice agents: an `end_of_turn` message when the speaker has finished their turn, judged from how they speak and not from silence alone. `true` uses the defaults (`GET /v2/live/defaults`), an object overrides some of them. Off when omitted. - `threshold` (number, default: `null`, 0 to 1): End-of-turn probability, between 0 and 1 (excluded), above which the turn ends. - `silence_ms` (integer, default: `null`, 0 to 10000): Silence, in milliseconds, before a turn is evaluated. - `stop_s` (number, default: `null`, 0 to 60): Silence, in seconds, that ends the turn whatever the probability. - `emit_scores` (boolean, default: `null`): Also send an `end_of_turn_score` message for each evaluation, to calibrate `threshold`. - `window_s` (number, default: `null`, 0 to 8): Audio, in seconds, an evaluation reads before the silence (at most 8). - `speech_dbfs` (number, default: `null`, -100 to 0): Level, in dBFS, under which audio counts as silence. - `reeval_ms` (integer, default: `null`, 0 to 10000): Interval, in milliseconds, between two evaluations of the same silence. - `numerals` (boolean or object, default: `false`): Spoken numbers, amounts and clock times written with digits in the final transcripts ("one thousand three hundred fifty five dollars" becomes "$1,355", "at nine fifteen" "at 9:15"); `text_raw` keeps the words. Where the turn router runs (the default; not with `turn_router: false`), it guides them: phone numbers are also written in their country's format ("06 12 34 56 78") and dates the language's way ("March 12, 2026"). `true` uses the countries of the language spoken, an object sets the country. - `region` (string, default: `null`): The country whose phone numbers and dates the transcripts are written in: an ISO 3166-1 alpha-2 code in capitals (`CH`, `CA`). Omitted: the countries where the language is spoken. - `named_entity_recognition` (boolean or object, default: `false`): Named entities (people, places, organizations, dates, amounts…) tagged in the final transcripts, and a `named_entity_recognition` message after each final. `true` tags every type. - `entities` (string or array of string or object): Which types: a preset (`pii`, `pci`, `phi`, `contact`, `financial`, `name`, `location`, `date`…), a list of types and presets, or `{"enable": [...], "disable": [...]}`. - `enable` (array of string) - `disable` (array of string) - `threshold` (number, default: `0.5`, 0 to 1): Confidence, from 0 to 1, under which an entity is ignored. - `pii` (boolean or object, default: `false`): Personal data (names, phone numbers, emails, card numbers…) redacted in the final transcripts, and a `pii` message after each final. With redaction on, no partial transcript is sent. - `entities` (string or array of string or object, default: `"pii"`): Which types to redact: a preset (`pii`, the default; `pci`, `phi`, `contact`, `financial`…), a list of types and presets, or `{"enable": [...], "disable": [...]}`. - `enable` (array of string) - `disable` (array of string) - `mode` (string, default: `"marker"`, one of: `marker`, `mask`, `tag`): `marker` replaces the data with an element (``), `mask` with mask characters inside the element, `tag` keeps the text inside the element. - `threshold` (number, default: `0.5`, 0 to 1): Confidence, from 0 to 1, under which a span is not redacted. - `mask_character` (string, default: `"*"`): The character of `mask` mode. - `coreference` (string, default: `"heuristics"`, one of: `heuristics`, `exact`, `off`): How mentions of the same entity get the same `id`: `heuristics` (the default), `exact` (same text only) or `off`. - `return_entities` (boolean, default: `false`): Also return the redacted text in the `pii` message (`text`). - `turn_router` (boolean or object): The number hold: an utterance stays open while a number, an amount or an address is still being dictated, so it is not cut at each pause. On by default where the model supports it; `false` turns it off. - `hold` (boolean): The number hold. Example: ```json { "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "messages_config": { "receive_partial_transcripts": true } } ``` ## Responses ### 201 The session is created. - `id` (string, required): Identifier of the session (a time-ordered UUID). - `created_at` (string, required): Creation date (ISO 8601, UTC). - `url` (string, required): WebSocket URL to stream the audio to. It carries a single-use token: connect within 60 seconds. Reconnecting to the same URL after a dropped connection resumes the session. ```json { "id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "created_at": "2026-10-12T09:04:17.210Z", "url": "wss://eu1.api.gladia.io/v2/live?token=eyJhbGciOi..." } ``` ### 400 The request is invalid: `validation_errors` lists why. ### 401 No API key, or an unknown one. ### 402 Your workspace has no credits left (`insufficient_credits`): new sessions are refused until credits are added; sessions already running are not stopped. ### 403 The key may not do this (its scopes, or the addresses it may be used from). ### 429 Your account already has its maximum number of open sessions. ### 500 No model in this region runs the session as asked (its model, languages and features together), or none has room right now: `no inference endpoint available`. ### 503 The service cannot answer right now: try again. ## Example ```bash curl -X POST https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" \ -H "content-type: application/json" \ -d '{ "encoding": "wav/pcm", "sample_rate": 16000, "bit_depth": 16, "channels": 1, "model": "solaria-3", "language_config": { "languages": [ "en" ] }, "messages_config": { "receive_partial_transcripts": true } }' ``` --- # Live WebSocket The `url` of `POST /v2/live`. Send the audio as binary frames, or as `audio_chunk` messages; send `stop_recording` when done. The server sends the messages below, then closes the connection. ## Server messages - `start_recording`: The session is ready: audio may be sent (`messages_config.receive_lifecycle_events`). (/api-reference/v2/live/message/start-recording.md) - `transcript`: A partial (`receive_partial_transcripts`) or final (`receive_final_transcripts`) transcript. (/api-reference/v2/live/message/transcript.md) - `speech_start`: Speech started on a channel (`receive_speech_events`). (/api-reference/v2/live/message/speech-start.md) - `speech_end`: Speech ended on a channel (`receive_speech_events`). (/api-reference/v2/live/message/speech-end.md) - `named_entity_recognition`: The entities of a final, right after it (`receive_realtime_processing_events`). (/api-reference/v2/live/message/named-entity-recognition.md) - `pii`: The personal data redacted in a final, right after it (with `pii`; on the WebSocket only). (/api-reference/v2/live/message/pii.md) - `sentiment_analysis`: The sentiment of a final, right after it (`receive_realtime_processing_events`). (/api-reference/v2/live/message/sentiment-analysis.md) - `translation`: One final translated into one target language (`receive_realtime_processing_events`). (/api-reference/v2/live/message/translation.md) - `end_of_turn`: The speaker finished their turn (`end_of_turn`): a voice agent may answer. (/api-reference/v2/live/message/end-of-turn.md) - `audio_chunk`: Audio transcribed: a client may drop it from its resend buffer (`receive_acknowledgments`). (/api-reference/v2/live/message/audio-chunk.md) - `stop_recording`: The answer to `stop_recording` (`receive_acknowledgments`). (/api-reference/v2/live/message/stop-recording.md) - `end_recording`: No more audio is accepted (`receive_lifecycle_events`). (/api-reference/v2/live/message/end-recording.md) - `post_transcript`: The whole session's transcript, after `end_recording` (`receive_post_processing_events`). (/api-reference/v2/live/message/post-transcript.md) - `post_final_transcript`: The session's result, the same as `GET /v2/live/{id}`'s (`receive_post_processing_events`). (/api-reference/v2/live/message/post-final-transcript.md) - `end_session`: The last message: the server closes the WebSocket after it (`receive_lifecycle_events`). (/api-reference/v2/live/message/end-session.md) - `error`: Something failed during the session (`receive_errors`). (/api-reference/v2/live/message/error.md) ## Close codes - `1000`: The session ended normally. - `1011`: The session failed; its reason says why. `rejected: …`: an option the model refused (a translation target it does not serve, translation with PII redaction…): fix the init body and create a new session. - `1012`: The service is restarting: reconnect to the same URL to resume. - `4401`: Invalid or expired token. --- # Get a session `GET /v2/live/{id}` A session's status and options and, once it ended, its transcript. Authentication: your API key in the `x-gladia-key` header. ## Parameters - `id` (path, string, required, uuid): Identifier of the session (`id` of `POST /v2/live`). ## Responses ### 200 The session. - `id` (string, required): Identifier of the session. - `request_id` (string, required): Identifier to quote to support. - `version` (2, default: `2`) - `status` (string, required, one of: `queued`, `processing`, `done`, `error`): `queued`: created, no WebSocket yet; `processing`: streaming (or dropped and still resumable); `done`: ended, the result is there; `error`: the session failed. - `created_at` (string, required): Creation date (ISO 8601, UTC). - `completed_at` (string, required): End date, once `done` or `error`. - `custom_metadata` (object, required): The init body's `custom_metadata`. - `error_code` (integer, required): `500` when the session failed, else `null`. - `kind` ("live", default: `"live"`) - `file` (object, required): The session's audio, once it ended. - `id` (string, required): Identifier of the session. - `filename` (string, required): Name of the recording, `null` when the audio was not recorded. - `source` (string, required): Download URL of the recording (`GET /v2/live/{id}/file`), `null` when not recorded. - `audio_duration` (number, required): Audio received, in seconds. - `number_of_channels` (integer, required) - `request_params` (object, required): The session's options, as validated. - `result` (object, required): The transcript, once `done`. - `metadata` (object, required) - `audio_duration` (number, required): Audio received, in seconds. - `billing_time` (number, required): Billed duration, in seconds. - `number_of_distinct_channels` (integer, required): Number of audio channels. - `transcription_time` (number, required): How long the session ran, in seconds. - `transcription` (object, required) - `languages` (array of string, required): Languages heard, in order of appearance. - `utterances` (array of object, required): Every final utterance, by start time. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `full_transcript` (string, required): The final utterances' text, joined. - `translation` (object, default: `null`): With `realtime_processing.translation`. - `success` (boolean, required) - `is_empty` (boolean, required): `true` when nothing was translated. - `exec_time` (number, required) - `error` (any, default: `null`) - `results` (array of object, required): One result per target language. - `error` (any, default: `null`): Always `null`. - `languages` (array of string, required): The target language of this result. - `full_transcript` (string, required): The translated text, joined. - `utterances` (array of object, required): The translated utterances, one per final. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `sentences` (array of any, required): Always empty. - `subtitles` (array of any, required): Always empty. - `messages` (array of any, required): The final transcript messages, as sent on the WebSocket. ```json { "id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "request_id": "G-0199d7a2", "version": 2, "status": "queued", "created_at": "string", "completed_at": "string", "custom_metadata": {}, "error_code": 0, "kind": "live", "file": { "id": "string", "filename": "string", "source": "string", "audio_duration": 0.5, "number_of_channels": 0 }, "request_params": {}, "result": { "metadata": { "audio_duration": 0.5, "billing_time": 0.5, "number_of_distinct_channels": 0, "transcription_time": 0.5 }, "transcription": { "languages": [ "string" ], "utterances": [ { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [], "text_raw": "string", "entities": [], "forced": "max_duration" } ], "full_transcript": "string" }, "translation": { "success": false, "is_empty": false, "exec_time": 0.5, "error": null, "results": [ { "error": null, "languages": [], "full_transcript": "string", "utterances": [], "sentences": [], "subtitles": [] } ] }, "messages": [ null ] } } ``` ### 401 No API key, or an unknown one. ### 403 The key may not do this (its scopes, or the addresses it may be used from). ### 404 No such session for your account. ### 503 The service cannot answer right now: try again. ## Example ```bash curl https://eu1.api.gladia.io/v2/live/0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b \ -H "x-gladia-key: $GLADIA_API_KEY" ``` --- # List sessions `GET /v2/live` Your account's sessions, newest first, one page at a time. Authentication: your API key in the `x-gladia-key` header. ## Parameters - `limit` (query, integer, default: `20`, 1 to 100): Sessions per page, at most 100. - `offset` (query, integer, default: `0`, 0 to 1000): Sessions to skip, at most 1000; past that, follow `next`. - `cursor` (query, string): Where the next page starts: set in `next`, never by hand. - `date` (query, string, date): Only the sessions created that day (`2026-10-12`). - `after_date` (query, string, date-time): Only the sessions created after that date (ISO 8601). - `before_date` (query, string, date-time): Only the sessions created before that date (ISO 8601). - `status` (query, array of string): Only the sessions with these statuses (repeat it for several). - `custom_metadata` (query, string): Only the sessions whose `custom_metadata` has these values: a JSON object (`{"user": "john"}`), or `custom_metadata[user]=john`. ## Responses ### 200 A page of sessions. - `first` (string, required): URL of the first page. - `current` (string, required): URL of this page. - `next` (string, required): URL of the next page (it carries a `cursor`), `null` on the last page. - `items` (array of object, required): The sessions, newest first. - `id` (string, required): Identifier of the session. - `request_id` (string, required): Identifier to quote to support. - `version` (2, default: `2`) - `status` (string, required, one of: `queued`, `processing`, `done`, `error`): `queued`: created, no WebSocket yet; `processing`: streaming (or dropped and still resumable); `done`: ended, the result is there; `error`: the session failed. - `created_at` (string, required): Creation date (ISO 8601, UTC). - `completed_at` (string, required): End date, once `done` or `error`. - `custom_metadata` (object, required): The init body's `custom_metadata`. - `error_code` (integer, required): `500` when the session failed, else `null`. - `kind` ("live", default: `"live"`) - `file` (object, required): The session's audio, once it ended. - `id` (string, required): Identifier of the session. - `filename` (string, required): Name of the recording, `null` when the audio was not recorded. - `source` (string, required): Download URL of the recording (`GET /v2/live/{id}/file`), `null` when not recorded. - `audio_duration` (number, required): Audio received, in seconds. - `number_of_channels` (integer, required) - `request_params` (object, required): The session's options, as validated. - `result` (object, required): The transcript, once `done`. - `metadata` (object, required) - `audio_duration` (number, required): Audio received, in seconds. - `billing_time` (number, required): Billed duration, in seconds. - `number_of_distinct_channels` (integer, required): Number of audio channels. - `transcription_time` (number, required): How long the session ran, in seconds. - `transcription` (object, required) - `languages` (array of string, required): Languages heard, in order of appearance. - `utterances` (array of object, required): Every final utterance, by start time. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `full_transcript` (string, required): The final utterances' text, joined. - `translation` (object, default: `null`): With `realtime_processing.translation`. - `success` (boolean, required) - `is_empty` (boolean, required): `true` when nothing was translated. - `exec_time` (number, required) - `error` (any, default: `null`) - `results` (array of object, required): One result per target language. - `error` (any, default: `null`): Always `null`. - `languages` (array of string, required): The target language of this result. - `full_transcript` (string, required): The translated text, joined. - `utterances` (array of object, required): The translated utterances, one per final. - `sentences` (array of any, required): Always empty. - `subtitles` (array of any, required): Always empty. - `messages` (array of any, required): The final transcript messages, as sent on the WebSocket. ```json { "first": "string", "current": "string", "next": "string", "items": [ { "id": "0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b", "request_id": "G-0199d7a2", "version": 2, "status": "queued", "created_at": "string", "completed_at": "string", "custom_metadata": {}, "error_code": 0, "kind": "live", "file": { "id": "string", "filename": "string", "source": "string", "audio_duration": 0.5, "number_of_channels": 0 }, "request_params": {}, "result": { "metadata": { "audio_duration": 0.5, "billing_time": 0.5, "number_of_distinct_channels": 0, "transcription_time": 0.5 }, "transcription": { "languages": [], "utterances": [], "full_transcript": "string" }, "translation": { "success": false, "is_empty": false, "exec_time": 0.5, "error": null, "results": [] }, "messages": [ null ] } } ] } ``` ### 400 The request is invalid: `validation_errors` lists why. ### 401 No API key, or an unknown one. ### 403 The key may not do this (its scopes, or the addresses it may be used from). ### 503 The service cannot answer right now: try again. ## Example ```bash curl https://eu1.api.gladia.io/v2/live \ -H "x-gladia-key: $GLADIA_API_KEY" ``` --- # Download the audio `GET /v2/live/{id}/file` The session's recording, in FLAC, for accounts that record their audio. Available about 40 seconds after the session ends. A `Range` header returns part of it. Authentication: your API key in the `x-gladia-key` header. ## Parameters - `id` (path, string, required, uuid): Identifier of the session (`id` of `POST /v2/live`). ## Responses ### 200 The recording. ### 206 The requested range of the recording. ### 302 A redirect to a signed download URL. ### 401 No API key, or an unknown one. ### 403 The key may not do this (its scopes, or the addresses it may be used from). ### 404 No such session for your account. ### 503 The service cannot answer right now: try again. ## Example ```bash curl -L https://eu1.api.gladia.io/v2/live/0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b/file \ -H "x-gladia-key: $GLADIA_API_KEY" \ -o session.flac ``` --- # Delete a session `DELETE /v2/live/{id}` Deletes a session that has ended, its transcript and its recording. Authentication: your API key in the `x-gladia-key` header. ## Parameters - `id` (path, string, required, uuid): Identifier of the session (`id` of `POST /v2/live`). ## Responses ### 202 Deleted. ### 401 No API key, or an unknown one. ### 403 The session is still running. ### 404 No such session for your account. ### 503 The service cannot answer right now: try again. ## Example ```bash curl -X DELETE https://eu1.api.gladia.io/v2/live/0199d7a2-5c3e-7b41-9f0a-3c2d1e4f5a6b \ -H "x-gladia-key: $GLADIA_API_KEY" ``` --- # Get the defaults `GET /v2/live/defaults` The value each option gets when the init body omits it. Authentication: your API key in the `x-gladia-key` header. ## Responses ### 200 The defaults in force. - `endpointing` (number, required): Silence, in seconds, that ends an utterance. - `maximum_duration_without_endpointing` (number, required): Longest utterance, in seconds, before a final is forced; `null`: no limit. - `end_of_turn` (object, required): The end-of-turn options a session starts from. - `threshold` (number, required): End-of-turn probability above which the turn ends. - `silence_ms` (integer, required): Silence, in milliseconds, before a turn is evaluated. - `stop_s` (number, required): Silence, in seconds, that ends the turn whatever the probability. - `reeval_ms` (integer, required): Interval, in milliseconds, between two evaluations of the same silence. - `window_s` (number, required): Audio, in seconds, the evaluation reads before the silence. - `speech_dbfs` (number, required): Level, in dBFS, under which audio counts as silence. - `turn_router` (object, required): The number hold a session gets when the model serving it supports it. - `hold` (boolean, required): An utterance stays open while a number, an amount or an address is still being dictated. - `emit` (boolean, required): Turn verdicts sent as messages. - `route_numerals` (boolean, required): Number words routed through the hold. ```json { "endpointing": 0.64, "maximum_duration_without_endpointing": 0.5, "end_of_turn": { "threshold": 0.5, "silence_ms": 0, "stop_s": 0.5, "reeval_ms": 0, "window_s": 0.5, "speech_dbfs": 0.5 }, "turn_router": { "hold": false, "emit": false, "route_numerals": false } } ``` ### 401 No API key, or an unknown one. ### 403 The key may not do this (its scopes, or the addresses it may be used from). ## Example ```bash curl https://eu1.api.gladia.io/v2/live/defaults \ -H "x-gladia-key: $GLADIA_API_KEY" ``` --- # start_recording message The session is ready: audio may be sent (`messages_config.receive_lifecycle_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("start_recording", required) ## Example ```json { "session_id": "string", "created_at": "string", "type": "start_recording" } ``` --- # transcript message A partial (`receive_partial_transcripts`) or final (`receive_final_transcripts`) transcript. ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("transcript", required) - `data` (object, required) - `id` (string, required): Identifier of the utterance: `_`, stable for the whole session. - `is_final` (boolean, required): `false`: a partial transcript, replaced by the next one; `true`: the final transcript of the utterance. - `utterance` (object, required): A piece of speech between two pauses, on one channel. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. ## Example ```json { "session_id": "string", "created_at": "string", "type": "transcript", "data": { "id": "00_00000003", "is_final": false, "utterance": { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" } } } ``` --- # speech_start message Speech started on a channel (`receive_speech_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("speech_start", required) - `data` (object, required) - `time` (number, required): When speech started or ended, in seconds. - `channel` (integer, required) ## Example ```json { "session_id": "string", "created_at": "string", "type": "speech_start", "data": { "time": 0.5, "channel": 0 } } ``` --- # speech_end message Speech ended on a channel (`receive_speech_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("speech_end", required) - `data` (object, required) - `time` (number, required): When speech started or ended, in seconds. - `channel` (integer, required) ## Example ```json { "session_id": "string", "created_at": "string", "type": "speech_end", "data": { "time": 0.5, "channel": 0 } } ``` --- # named_entity_recognition message The entities of a final, right after it (`receive_realtime_processing_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("named_entity_recognition", required) - `error` (object, default: `null`) - `message` (string, required) - `data` (object, required) - `utterance_id` (string, required) - `utterance` (object, required): A piece of speech between two pauses, on one channel. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `results` (array of object, required) - `entity_type` (string, required): The entity's type, for example `name`. - `text` (string, required): The entity as heard. - `start` (number, required): Start, in seconds. - `end` (number, required): End, in seconds. - `id` (integer, default: `null`): The entity's number in the session: the same person is always the same number. - `score` (number, default: `null`): Confidence, from 0 to 1. - `char_start` (integer, default: `null`): Offset of the entity's element in the final's `text`. - `char_end` (integer, default: `null`): End offset of the entity's element in the final's `text`. ## Example ```json { "session_id": "string", "created_at": "string", "type": "named_entity_recognition", "error": { "message": "string" }, "data": { "utterance_id": "string", "utterance": { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" }, "results": [ { "entity_type": "string", "text": "string", "start": 0.5, "end": 0.5, "id": 0, "score": 0.5, "char_start": 0, "char_end": 0 } ] } } ``` --- # pii message The personal data redacted in a final, right after it (with `pii`; on the WebSocket only). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("pii", required) - `error` (object, default: `null`) - `message` (string, required) - `data` (object, required) - `utterance_id` (string, required) - `utterance` (object, required): A piece of speech between two pauses, on one channel. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `results` (array of object, required) - `entity_type` (string, required): The kind of personal data, for example `phone_number`. - `id` (integer, default: `null`): Its number in the session. - `action` (string, default: `null`): What was done: masked or tagged. - `char_start` (integer, default: `null`) - `char_end` (integer, default: `null`) - `text` (string, default: `null`): The data as heard, only with `pii.return_entities`. ## Example ```json { "session_id": "string", "created_at": "string", "type": "pii", "error": { "message": "string" }, "data": { "utterance_id": "string", "utterance": { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" }, "results": [ { "entity_type": "string", "id": 0, "action": "string", "char_start": 0, "char_end": 0, "text": "string" } ] } } ``` --- # sentiment_analysis message The sentiment of a final, right after it (`receive_realtime_processing_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("sentiment_analysis", required) - `error` (object, default: `null`) - `message` (string, required) - `data` (object, required) - `utterance_id` (string, required) - `utterance` (object, required): A piece of speech between two pauses, on one channel. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `results` (array of object, required) - `sentiment` (string, required, one of: `positive`, `neutral`, `negative`) - `emotion` (string, required): The dominant emotion, for example `joy`; empty when none. - `text` (string, required) - `start` (number, required) - `end` (number, required) - `channel` (integer, required) ## Example ```json { "session_id": "string", "created_at": "string", "type": "sentiment_analysis", "error": { "message": "string" }, "data": { "utterance_id": "string", "utterance": { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" }, "results": [ { "sentiment": "positive", "emotion": "string", "text": "string", "start": 0.5, "end": 0.5, "channel": 0 } ] } } ``` --- # translation message One final translated into one target language (`receive_realtime_processing_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("translation", required) - `error` (any, default: `null`) - `data` (object, required) - `utterance_id` (string, required) - `utterance` (object or object, required) - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `original_language` (string, required) - `target_language` (string, required) - `translated_utterance` (object, required): `null` when the translation failed (see `error`). - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. ## Example ```json { "session_id": "string", "created_at": "string", "type": "translation", "error": null, "data": { "utterance_id": "string", "utterance": { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" }, "original_language": "string", "target_language": "string", "translated_utterance": { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" } } } ``` --- # end_of_turn message The speaker finished their turn (`end_of_turn`): a voice agent may answer. ## Fields - `type` ("end_of_turn", required) - `session_id` (string, required) - `data` (object, required) - `channel` (integer, required) - `probability` (number, required): Probability that the speaker finished. - `reason` (string, required, one of: `model`, `timeout`): `model`: the end-of-turn probability crossed `threshold`; `timeout`: the silence reached `stop_s`. - `end` (number, required): When the turn ended, in seconds. - `speech_duration` (number, required): Speech in the turn, in seconds. ## Example ```json { "type": "end_of_turn", "session_id": "string", "data": { "channel": 0, "probability": 0.5, "reason": "model", "end": 0.5, "speech_duration": 0.5 } } ``` --- # audio_chunk message Audio transcribed: a client may drop it from its resend buffer (`receive_acknowledgments`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("audio_chunk", required) - `acknowledged` (boolean, required) - `error` (any, default: `null`) - `data` (object, required) - `byte_range` (array of integer, required, at least 2 item(s)): First and last byte (excluded) of the acknowledged audio, from the start of the session. - `time_range` (array of integer, required, at least 2 item(s)): The same range in milliseconds. ## Example ```json { "session_id": "string", "created_at": "string", "type": "audio_chunk", "acknowledged": false, "error": null, "data": { "byte_range": [ 0 ], "time_range": [ 0 ] } } ``` --- # stop_recording message The answer to `stop_recording` (`receive_acknowledgments`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("stop_recording", required) - `acknowledged` (boolean, required) - `error` (any, default: `null`) - `data` (object, required) - `recording_duration` (number, required): Audio received, in seconds. - `recording_left_to_process` (number, required): Audio, in seconds, still being transcribed. ## Example ```json { "session_id": "string", "created_at": "string", "type": "stop_recording", "acknowledged": false, "error": null, "data": { "recording_duration": 0.5, "recording_left_to_process": 0.5 } } ``` --- # end_recording message No more audio is accepted (`receive_lifecycle_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("end_recording", required) - `data` (object, required) - `recording_duration` (number, required): Audio received, in seconds. ## Example ```json { "session_id": "string", "created_at": "string", "type": "end_recording", "data": { "recording_duration": 0.5 } } ``` --- # post_transcript message The whole session's transcript, after `end_recording` (`receive_post_processing_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("post_transcript", required) - `data` (object, required) - `languages` (array of string, required): Languages heard, in order of appearance. - `utterances` (array of object, required): Every final utterance, by start time. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `full_transcript` (string, required): The final utterances' text, joined. ## Example ```json { "session_id": "string", "created_at": "string", "type": "post_transcript", "data": { "languages": [ "string" ], "utterances": [ { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [ { "word": " hello", "start": 0.5, "end": 0.5, "confidence": 0.5, "speaker": 0 } ], "text_raw": "string", "entities": [ { "label": "string", "id": 0, "action": "string", "score": 0.5, "text": "string", "start": 0, "end": 0 } ], "forced": "max_duration" } ], "full_transcript": "string" } } ``` --- # post_final_transcript message The session's result, the same as `GET /v2/live/{id}`'s (`receive_post_processing_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("post_final_transcript", required) - `data` (object, required) - `metadata` (object, required) - `audio_duration` (number, required): Audio received, in seconds. - `billing_time` (number, required): Billed duration, in seconds. - `number_of_distinct_channels` (integer, required): Number of audio channels. - `transcription_time` (number, required): How long the session ran, in seconds. - `transcription` (object, required) - `languages` (array of string, required): Languages heard, in order of appearance. - `utterances` (array of object, required): Every final utterance, by start time. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `word` (string, required): The word, with its leading space when it follows another. - `start` (number, required): Start of the word, in seconds from the start of the audio. - `end` (number, required): End of the word, in seconds from the start of the audio. - `confidence` (number, required): Confidence, from 0 to 1. - `speaker` (integer, default: `null`): Speaker of the word, with `diarization`. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `label` (string, default: `null`): Entity type, for example `name`, `phone_number`, `location_city`. - `id` (integer, default: `null`): The entity's number in the session, per type: the same person is always the same number. - `action` (string, default: `null`): `ner`: tagged; otherwise how the personal data was redacted. - `score` (number, default: `null`): Confidence, from 0 to 1. - `text` (string, default: `null`): The entity as heard (never sent for masked personal data unless `pii.return_entities`). - `start` (integer, default: `null`): Offset of the entity's element in `text`, in characters. - `end` (integer, default: `null`): End offset of the entity's element in `text`, in characters. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `full_transcript` (string, required): The final utterances' text, joined. - `translation` (object, default: `null`) - `success` (boolean, required) - `is_empty` (boolean, required): `true` when nothing was translated. - `exec_time` (number, required) - `error` (any, default: `null`) - `results` (array of object, required): One result per target language. - `error` (any, default: `null`): Always `null`. - `languages` (array of string, required): The target language of this result. - `full_transcript` (string, required): The translated text, joined. - `utterances` (array of object, required): The translated utterances, one per final. - `text` (string, required): The transcript. With named entity recognition or PII redaction on, it carries their inline elements (`…`, ``). - `start` (number, required): Start, in seconds from the start of the audio. - `end` (number, required): End, in seconds from the start of the audio. - `language` (string, required): Language spoken (ISO 639-1). - `confidence` (number, required): Confidence, from 0 to 1. - `channel` (integer, required): Audio channel the utterance was heard on, from 0. - `speaker` (integer, default: `null`): Speaker number, with `diarization`. - `words` (array of object, required): The words with their timestamps. - `text_raw` (string, default: `null`): The transcript without inline elements (with named entity recognition; never with PII redaction). - `entities` (array of object, default: `null`): The entities of the utterance, with named entity recognition or PII redaction. - `forced` ("max_duration", default: `null`): Set when the utterance was closed by `maximum_duration_without_endpointing` rather than by a pause. - `sentences` (array of any, required): Always empty. - `subtitles` (array of any, required): Always empty. ## Example ```json { "session_id": "string", "created_at": "string", "type": "post_final_transcript", "data": { "metadata": { "audio_duration": 0.5, "billing_time": 0.5, "number_of_distinct_channels": 0, "transcription_time": 0.5 }, "transcription": { "languages": [ "string" ], "utterances": [ { "text": "Hello, how can I help you?", "start": 0.5, "end": 0.5, "language": "en", "confidence": 0.5, "channel": 0, "speaker": 0, "words": [], "text_raw": "string", "entities": [], "forced": "max_duration" } ], "full_transcript": "string" }, "translation": { "success": false, "is_empty": false, "exec_time": 0.5, "error": null, "results": [ { "error": null, "languages": [], "full_transcript": "string", "utterances": [], "sentences": [], "subtitles": [] } ] } } } ``` --- # end_session message The last message: the server closes the WebSocket after it (`receive_lifecycle_events`). ## Fields - `session_id` (string, required): Identifier of the session. - `created_at` (string, required): When the message was sent (ISO 8601, UTC). - `type` ("end_session", required) ## Example ```json { "session_id": "string", "created_at": "string", "type": "end_session" } ``` --- # error message Something failed during the session (`receive_errors`). ## Fields - `type` ("error", required) - `session_id` (string, default: `null`) - `error` (any, default: `null`): What failed. ## Example ```json { "type": "error", "session_id": "string", "error": null } ```