Skip to content
Gladia Enterprise API

Named entity recognition

The problem. A caller gives their name, their order number and a delivery date; your CRM needs those fields, not a paragraph. Named entity recognition finds them in each final transcript as it arrives: people, organizations, places, dates, amounts, emails, phone numbers and about 70 types in all.

Where it works. Every model.

Step by step

Section titled “Step by step”

1. Turn it on

Section titled “1. Turn it on”
{ "named_entity_recognition": true }

realtime_processing.named_entity_recognition: true, Gladia’s field, does the same.

2. Start a session with it

Section titled “2. Start a session with it”

Each program streams a 16-bit WAV file and prints what comes back. Put your API key in the GLADIA_API_KEY environment variable, then run python entities.py meeting.wav.

entities.py
# pip install gladiaio-sdk requests
import os
import sys
import threading
import wave


import requests
from gladiaio_sdk import GladiaClient, LiveV2ConnectSessionOptions


API_URL = os.environ.get("GLADIA_API_URL", "https://eu1.api.gladia.io")
API_KEY = os.environ["GLADIA_API_KEY"]


with wave.open(sys.argv[1]) as wav:  # a 16-bit PCM WAV file
    rate, channels = wav.getframerate(), wav.getnchannels()
    audio = wav.readframes(wav.getnframes())


config = {
    "encoding": "wav/pcm",
    "sample_rate": rate,
    "bit_depth": 16,
    "channels": channels,
    "model": "solaria-3",
    "language_config": {
        "languages": [
            "en"
        ]
    },
    "named_entity_recognition": {
        "entities": [
            "name",
            "organization",
            "date",
            "phone_number"
        ]
    }
}


client = GladiaClient(api_key=API_KEY, api_url=API_URL)
# `named_entity_recognition` with options is a Gladia Enterprise option the Python SDK does not send yet
# (gladiaio/sdk#61): create the session over HTTP, then let the SDK stream it.
created = requests.post(f"{API_URL}/v2/live", headers={"x-gladia-key": API_KEY}, json=config, timeout=10)
created.raise_for_status()
info = created.json()
session = client.live_v2().connect_session(LiveV2ConnectSessionOptions(id=info["id"], url=info["url"]))
ended, errors = threading.Event(), []


@session.on("message")
def on_message(message):
    if message.type == "transcript" and message.data.is_final:
        print(message.data.utterance.text)
    elif message.type == "named_entity_recognition":
        for entity in message.data.results:
            print(f"  {entity.entity_type}: {entity.text}")


session.on("error", errors.append)
session.once("ended", lambda _: ended.set())


step = rate * channels * 2 // 10  # 100 ms of audio per chunk
for start in range(0, len(audio), step):
    session.send_audio(audio[start : start + step])  # live audio: send each chunk as it is captured
session.stop_recording()


ended.wait(timeout=60)
sys.exit(1 if errors or not ended.is_set() else 0)

3. Read the entities

Section titled “3. Read the entities”

The final’s text carries them inline, and text_raw keeps the plain sentence:

{
  "type": "transcript",
  "data": {
    "id": "00_00000002",
    "is_final": true,
    "utterance": {
      "text": "Hi, this is <ner:name id=\"1\">Marie Curie</ner:name> from <ner:organization id=\"1\">Gladia</ner:organization>.",
      "text_raw": "Hi, this is Marie Curie from Gladia.",
      "entities": [
        { "label": "name", "id": 1, "action": "ner", "text": "Marie Curie", "start": 12, "end": 52 },
        { "label": "organization", "id": 1, "action": "ner", "text": "Gladia", "start": 58, "end": 102 }
      ]
    }
  }
}

Right after it, a named_entity_recognition message lists them with their times in the audio:

{
  "type": "named_entity_recognition",
  "error": null,
  "data": {
    "utterance_id": "00_00000002",
    "results": [
      { "entity_type": "name", "text": "Marie Curie", "start": 4.9, "end": 5.6, "id": 1, "score": 0.97, "char_start": 12, "char_end": 52 }
    ]
  }
}

Options

Section titled “Options”
option what it does
entities Which types: a preset (name, location, date, contact, financial, identifiers, pii…), a list of types and presets, or {"enable": [...], "disable": [...]}. Omitted: every type.
threshold Confidence, from 0 to 1, under which an entity is ignored. Default 0.5.

A parent type (name, location, date) also selects its children (name_given, name_family, location_city…). An unknown type is refused, with the list of known ones.

Limits and gotchas

Section titled “Limits and gotchas”
  • The text changes. With entities on, utterance.text contains <ner:…> elements. Display text_raw, or render the elements; never show text as is to an end user.
  • Finals only. Partials are not tagged.
  • id follows the entity through the session. The same person keeps id 1, even when only their first name is heard again. start/end on the utterance and char_start/char_end in the message locate the element in text, not in text_raw.
  • The message is not stored. The named_entity_recognition messages are not part of the session’s result; the entities of each utterance are. Store the messages yourself (WebSocket or callback) if you need their audio times.
  • If tagging fails on a final, the message comes with error set and that final has no words. Rare; the session goes on.
  • Personal data stays visible. Entities are tagged, not hidden: use PII redaction to remove them. Both together: the types selected for redaction are redacted, the others tagged.

Troubleshooting

Section titled “Troubleshooting”
symptom cause fix
users see <ner:name id="1"> in your interface text displayed as is display text_raw, or render the elements
no named_entity_recognition message messages_config.receive_realtime_processing_events: false turn it on
rejected: … unknown entity labels (WebSocket 1011) a type that does not exist use the types and presets of the error message
too many false entities low confidence spans raise threshold