Sentiment analysis

The problem. A customer is getting angry and nobody notices before the call goes wrong. Sentiment analysis rates each final sentence as positive, neutral or negative, with its dominant emotion, while the call runs: escalate to a supervisor, flag calls to review, chart a conversation’s mood.

Where it works. Every model. Best in English, French, Spanish, German, Italian, Portuguese, Dutch, Polish, Turkish, Russian, Japanese, Chinese and Arabic.

Use the SDK

The Gladia SDKs stream the audio, keep it until the API has transcribed it, and reconnect after a network drop without losing a word. Most integration issues come from hand-written WebSocket clients: start from the SDK examples below.

Step by step

1. Turn it on

{ "realtime_processing": { "sentiment_analysis": true } }

2. Start a session with it

Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the GLADIA_API_KEY environment variable, then run python sentiment.py meeting.wav.

Python SDK

sentiment.py

# pip install gladiaio-sdk
import os
import sys
import threading
import wave

from gladiaio_sdk import GladiaClient, LiveV2InitRequest

API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io")
API_KEY = os.environ["GLADIA_API_KEY"]

with wave.open(sys.argv[1]) as wav:  # a 16-bit PCM WAV file
    rate, channels = wav.getframerate(), wav.getnchannels()
    audio = wav.readframes(wav.getnframes())

config = {
    "encoding": "wav/pcm",
    "sample_rate": rate,
    "bit_depth": 16,
    "channels": channels,
    "model": "solaria-3",
    "language_config": {
        "languages": [
            "en"
        ]
    },
    "realtime_processing": {
        "sentiment_analysis": True
    }
}

client = GladiaClient(api_key=API_KEY, api_url=API_URL)
session = client.live_v2().start_session(LiveV2InitRequest.from_dict(config))
ended, errors = threading.Event(), []

@session.on("message")
def on_message(message):
    if message.type == "transcript" and message.data.is_final:
        print(message.data.utterance.text)
    elif message.type == "sentiment_analysis":
        for result in message.data.results:
            print(f"  {result.sentiment} ({result.emotion})")

session.on("error", errors.append)
session.once("ended", lambda _: ended.set())

step = rate * channels * 2 // 10  # 100 ms of audio per chunk
for start in range(0, len(audio), step):
    session.send_audio(audio[start : start + step])  # live audio: send each chunk as it is captured
session.stop_recording()

ended.wait(timeout=60)
sys.exit(1 if errors or not ended.is_set() else 0)

JavaScript SDK

sentiment.mjs

// npm install @gladiaio/sdk
import { readFileSync } from "node:fs";
import { GladiaClient } from "@gladiaio/sdk";

const { sampleRate, channels, pcm } = readWav(process.argv[2]); // a 16-bit PCM WAV file

const config = {
  encoding: "wav/pcm",
  sample_rate: sampleRate,
  bit_depth: 16,
  channels,
  model: "solaria-3",
  language_config: {
    languages: [
      "en"
    ]
  },
  realtime_processing: {
    sentiment_analysis: true
  }
};

const client = new GladiaClient({
  apiKey: process.env.GLADIA_API_KEY,
  apiUrl: process.env.GLADIA_API_URL ?? "https://eu1.api.gladia.io",
});
const session = client.liveV2().startSession(config);

session.on("message", (message) => {
  if (message.type === "transcript" && message.data.is_final) console.log(message.data.utterance.text);
  if (message.type === "sentiment_analysis") {
    for (const { sentiment, emotion } of message.data.results) console.log(`  ${sentiment} (${emotion})`);
  }
});
session.on("error", (error) => {
  console.error(error);
  process.exitCode = 1;
});
session.once("ended", () => process.exit());

const step = (sampleRate * channels * 2) / 10; // 100 ms of audio per chunk
for (let start = 0; start < pcm.length; start += step) {
  session.sendAudio(pcm.subarray(start, start + step)); // live audio: send each chunk as it is captured
}
session.stopRecording();

function readWav(path) {
  const file = readFileSync(path);
  let fmt, pcm;
  for (let pos = 12; pos < file.length; ) {
    const id = file.toString("ascii", pos, pos + 4);
    const size = file.readUInt32LE(pos + 4);
    if (id === "fmt ") fmt = { channels: file.readUInt16LE(pos + 10), sampleRate: file.readUInt32LE(pos + 12) };
    if (id === "data") pcm = file.subarray(pos + 8, pos + 8 + size);
    pos += 8 + size + (size % 2);
  }
  return { ...fmt, pcm };
}

HTTP

Create the session, then stream its audio to the returned url (Live WebSocket):

curl -X POST https://eu1.api.gladia.io/v2/live \
  -H "x-gladia-key: $GLADIA_API_KEY" \
  -H "content-type: application/json" \
  -d '{
    "encoding": "wav/pcm",
    "sample_rate": 16000,
    "bit_depth": 16,
    "channels": 1,
    "model": "solaria-3",
    "language_config": {
      "languages": [
        "en"
      ]
    },
    "realtime_processing": {
      "sentiment_analysis": true
    }
  }'

3. Read the sentiment

After each final comes a sentiment_analysis message:

{
  "type": "sentiment_analysis",
  "error": null,
  "data": {
    "utterance_id": "01_00000007",
    "utterance": { "text": "This is the third time I call about it.", "start": 41.2, "end": 43.6, "language": "en", "channel": 1 },
    "results": [
      { "sentiment": "negative", "emotion": "anger", "text": "This is the third time I call about it.", "start": 41.2, "end": 43.6, "channel": 1 }
    ]
  }
}

sentiment is positive, neutral or negative. emotion is one of: adoration, amusement, anger, awe, confusion, contempt, contentment, desire, disappointment, disgust, distress, ecstatic, elation, embarrassment, fear, interest, negative surprise, neutral, pain, positive surprise, realization, relief, sadness, sympathy, triumph (as snake_case), or empty.

Limits and gotchas

• It needs the number hold. Sentiment comes from the same analysis as the number hold: with turn_router: false, no sentiment_analysis message is sent, and no error either.
• One sentence at a time. Each final is rated on its own: a call’s overall mood is yours to aggregate.
• Finals only. Partials are not rated.
• Not stored. sentiment_analysis messages are not part of the session’s result: keep them as they arrive, on the WebSocket or with a callback (callback_config.receive_realtime_processing_events).
• Other languages get a rating too, less reliable.

Troubleshooting

symptom	cause	fix
no sentiment_analysis message	turn_router: false, or receive_realtime_processing_events: false	remove turn_router: false; keep the realtime events on
sentiment missing in GET /v2/live/{id}	not stored with the result	store the messages as they arrive