Live WebSocket

The url of POST /v2/live. Send the audio as binary frames, or as audio_chunk messages; send stop_recording when done. The server sends the messages below, then closes the connection.

Connect

Create the session with POST /v2/live, then open its url. The URL carries a single-use token, valid 60 seconds: create the session on your server and hand only the URL to browsers or phones, your API key never leaves your server. Reconnecting to the same URL after a dropped connection resumes the session (Reconnections).

Whole programs, with the SDKs and without: Quickstart. Try it with your microphone on the Playground.

Send

Audio as binary frames: raw samples in the session's encoding, little-endian, channels interleaved, any size (100 ms is typical). Or as JSON:

audio_chunk

Audio, as JSON. Binary frames of raw audio are accepted too, and cost a third less.

• type "audio_chunk" required
• data object required
  1 fields

  • chunk string required

    Audio, base64-encoded, in the session's encoding.

{"type":"audio_chunk","data":{"chunk":"string"}}

stop_recording

No more audio: the server sends the last finals, the post-processing results, then ends the session.

• type "stop_recording" required

{"type":"stop_recording"}

Receive

JSON messages, each with a type. messages_config in the init body chooses which ones come; the same messages can go to your callback.

type	what
start_recording	The session is ready: audio may be sent (messages_config.receive_lifecycle_events).
transcript	A partial (receive_partial_transcripts) or final (receive_final_transcripts) transcript.
speech_start	Speech started on a channel (receive_speech_events).
speech_end	Speech ended on a channel (receive_speech_events).
named_entity_recognition	The entities of a final, right after it (receive_realtime_processing_events).
pii	The personal data redacted in a final, right after it (with pii; on the WebSocket only).
sentiment_analysis	The sentiment of a final, right after it (receive_realtime_processing_events).
translation	One final translated into one target language (receive_realtime_processing_events).
end_of_turn	The speaker finished their turn (end_of_turn): a voice agent may answer.
audio_chunk	Audio transcribed: a client may drop it from its resend buffer (receive_acknowledgments).
stop_recording	The answer to stop_recording (receive_acknowledgments).
end_recording	No more audio is accepted (receive_lifecycle_events).
post_transcript	The whole session's transcript, after end_recording (receive_post_processing_events).
post_final_transcript	The session's result, the same as GET /v2/live/{id}'s (receive_post_processing_events).
end_session	The last message: the server closes the WebSocket after it (receive_lifecycle_events).
error	Something failed during the session (receive_errors).

Close codes

code	meaning
1000	The session ended normally.
1011	The session failed; its reason says why. rejected: …: an option the model refused (a translation target it does not serve, translation with PII redaction…): fix the init body and create a new session.
1012	The service is restarting: reconnect to the same URL to resume.
4401	Invalid or expired token.