elevenlabs/skills

speech-to-text

Transcribe audio to text using ElevenLabs Scribe v2.

View source
Original skill document

Rendered from the source repository. Headings, examples, code, tables, links, and referenced images are preserved.

ElevenLabs Speech-to-Text

Transcribe audio to text with Scribe v2 - supports 90+ languages, speaker diarization, and word-level timestamps.

Setup: See Installation Guide. For JavaScript, use @elevenlabs/* packages only.

Quick Start

Python

python
from elevenlabs import ElevenLabs

client = ElevenLabs()

with open("audio.mp3", "rb") as audio_file:
    result = client.speech_to_text.convert(file=audio_file, model_id="scribe_v2")

print(result.text)

JavaScript

javascript
import { ElevenLabsClient } from "@elevenlabs/elevenlabs-js";
import { createReadStream } from "fs";

const client = new ElevenLabsClient();
const result = await client.speechToText.convert({
  file: createReadStream("audio.mp3"),
  modelId: "scribe_v2",
});
console.log(result.text);

CLI

bash
elevenlabs speech-to-text convert --file audio.mp3 --model-id scribe_v2

Models

Model IDDescriptionBest For
scribe_v2State-of-the-art accuracy, 90+ languagesBatch transcription, subtitles, long-form audio
scribe_v2_medicalSpecialized recognition for medical and clinical audio, 90+ languagesClinical documentation, medical dictation, patient calls
scribe_v2_realtimeLow latency (~150ms)Live transcription, voice agents
scribe_v2_realtime_turboRealtime transcription variantLive transcription
scribe_v2_realtime_liteRealtime transcription variantLive transcription

Transcription with Timestamps

Word-level timestamps include type classification and speaker identification:

python
result = client.speech_to_text.convert(
    file=audio_file, model_id="scribe_v2", timestamps_granularity="word"
)

for word in result.words:
    print(f"{word.text}: {word.start}s - {word.end}s (type: {word.type})")

Speaker Diarization

Identify WHO said WHAT - the model labels each word with a speaker ID, useful for meetings, interviews, or any multi-speaker audio:

python
result = client.speech_to_text.convert(
    file=audio_file,
    model_id="scribe_v2",
    diarize=True
)

for word in result.words:
    print(f"[{word.speaker_id}] {word.text}")

For call recordings, the batch API can label diarized speakers as agent and customer by setting detect_speaker_roles=true alongside diarize=true. This option is not compatible with use_multi_channel=true.

If your workspace has registered speaker profiles, set use_speaker_library=true with diarize=true to match detected speakers against the speaker library.

bash
elevenlabs speech-to-text convert \
  --file call.mp3 \
  --model-id scribe_v2 \
  --diarize true \
  --detect-speaker-roles true \
  --use-speaker-library true

Multichannel Audio

Use use_multi_channel=true when each speaker is isolated on a separate audio channel. By default, the API returns one transcript per channel under transcripts; set multichannel_output_style="combined" to receive one transcript merged by timestamp, with channel_index on each word.

python
result = client.speech_to_text.convert(
    file=audio_file,
    model_id="scribe_v2",
    use_multi_channel=True,
    multichannel_output_style="combined",
)

Keyterm Prompting

Help the model recognize specific words it might otherwise mishear - product names, technical jargon, or unusual spellings (up to 100 terms):

python
result = client.speech_to_text.convert(
    file=audio_file,
    model_id="scribe_v2",
    keyterms=["ElevenLabs", "Scribe", "API"]
)

Language Detection

Automatic detection with optional language hint:

python
result = client.speech_to_text.convert(
    file=audio_file,
    model_id="scribe_v2",
    language_code="eng"  # ISO 639-1 or ISO 639-3 code
)

print(f"Detected: {result.language_code} ({result.language_probability:.0%})")

Supported Formats

Audio: MP3, WAV, M4A, FLAC, OGG, WebM, AAC, AIFF, Opus Video: MP4, AVI, MKV, MOV, WMV, FLV, WebM, MPEG, 3GPP

Limits: Up to 5.0GB file size, 10 hours duration

Response Format

json
{
  "text": "The full transcription text",
  "language_code": "eng",
  "language_probability": 0.98,
  "words": [
    {"text": "The", "start": 0.0, "end": 0.15, "type": "word", "speaker_id": "speaker_0"},
    {"text": " ", "start": 0.15, "end": 0.16, "type": "spacing", "speaker_id": "speaker_0"}
  ]
}

Word types:

  • word - An actual spoken word
  • spacing - Whitespace between words (useful for precise timing)
  • audio_event - Non-speech sounds the model detected (laughter, applause, music, etc.)

Error Handling

python
try:
    result = client.speech_to_text.convert(file=audio_file, model_id="scribe_v2")
except Exception as e:
    print(f"Transcription failed: {e}")

Common errors:

  • 401: Invalid API key
  • 422: Invalid parameters
  • 429: Rate limit exceeded

Tracking Costs

Monitor usage via request-id response header:

python
response = client.speech_to_text.with_raw_response.convert(file=audio_file, model_id="scribe_v2")
result = response.data
print(f"Request ID: {response.headers.get('request-id')}")

Real-Time Streaming

For live transcription with ultra-low latency (~150ms), use the real-time API. The real-time API produces two types of transcripts:

  • Partial transcripts: Interim results that update frequently as audio is processed - use these for live feedback (e.g., showing text as the user speaks)
  • Committed transcripts: Final, stable results after you "commit" - use these as the source of truth for your application

A "commit" tells the model to finalize the current segment. You can commit manually (e.g., when the user pauses) or use Voice Activity Detection (VAD) to auto-commit on silence.

Python (Server-Side)

python
import asyncio
from elevenlabs import ElevenLabs

client = ElevenLabs()

async def transcribe_realtime():
    async with client.speech_to_text.realtime.connect(
        model_id="scribe_v2_realtime",
        include_timestamps=True,
        keyterms=["ElevenLabs", "Scribe"],
        no_verbatim=True,
    ) as connection:
        await connection.stream_url("https://example.com/audio.mp3")

        async for event in connection:
            if event.type == "partial_transcript":
                print(f"Partial: {event.text}")
            elif event.type == "committed_transcript":
                print(f"Final: {event.text}")

asyncio.run(transcribe_realtime())

JavaScript (Client-Side with React)

typescript
import { useScribe, CommitStrategy } from "@elevenlabs/react";

function TranscriptionComponent() {
  const [transcript, setTranscript] = useState("");

  const scribe = useScribe({
    modelId: "scribe_v2_realtime",
    commitStrategy: CommitStrategy.VAD, // Auto-commit on silence for mic input
    keyterms: ["ElevenLabs", "Scribe"],
    noVerbatim: true,
    includeLanguageDetection: true,
    onPartialTranscript: (data) => console.log("Partial:", data.text),
    onCommittedTranscript: (data) => setTranscript((prev) => prev + data.text),
  });

  const start = async () => {
    // Get token from your backend (never expose API key to client)
    const { token } = await fetch("/scribe-token").then((r) => r.json());

    await scribe.connect({
      token,
      microphone: { echoCancellation: true, noiseSuppression: true },
    });
  };

  return <button onClick={start}>Start Recording</button>;
}

Commit Strategies

StrategyDescription
ManualYou call commit() when ready - use for file processing or when you control the audio segments
VADVoice Activity Detection auto-commits when silence is detected - use for live microphone input

Set includeLanguageDetection: true to receive the detected language code in delayed final transcript events.

typescript
// React: set commitStrategy on the hook (recommended for mic input)
import { useScribe, CommitStrategy } from "@elevenlabs/react";

const scribe = useScribe({
  modelId: "scribe_v2_realtime",
  commitStrategy: CommitStrategy.VAD,
  keyterms: ["ElevenLabs", "Scribe"],
  noVerbatim: true,
  // Optional VAD tuning:
  vadSilenceThresholdSecs: 1.5,
  vadThreshold: 0.4,
});
javascript
// JavaScript client: pass vad config on connect
const connection = await client.speechToText.realtime.connect({
  modelId: "scribe_v2_realtime",
  keyterms: ["ElevenLabs", "Scribe"],
  noVerbatim: true,
  vad: {
    silenceThresholdSecs: 1.5,
    threshold: 0.4,
  },
});

Event Types

EventDescription
partial_transcriptLive interim results
final_transcriptStable segment result sent before the segment is committed
final_transcript_with_timestampsDelayed final result with timestamps and/or detected language
committed_transcriptFinal results after commit
committed_transcript_with_timestampsFinal with word timing
committed_transcript_entitiesEntities detected in a committed segment
edited_transcriptOriginal and edited text for a committed segment when transcript editing is enabled
invalid_requestConnection parameters were rejected and the session closes
errorError occurred

See real-time references for complete documentation.

References

from this repository

More skills

All skills
elevenlabs
Official

setup-api-key

Guides users through setting up an ElevenLabs API key for REST API and SDK workflows. Use when the user needs to configure an ElevenLabs API key or when an API or SDK request fails because a key is missing. First checks whether ELEVENLABSAPIKEY is already configured and valid, and only runs full setup when needed.

installs
4
GitHub stars
484
Updated
Oct 9
elevenlabs
Official

sound-effects

Generate sound effects from text descriptions using ElevenLabs. Use when creating sound effects, generating audio textures, producing ambient sounds, cinematic impacts, UI sounds, or any audio that isn't speech. Supports looping, duration control, and prompt influence tuning.

installs
4
GitHub stars
484
Updated
Oct 9
elevenlabs
Official

text-to-speech

Convert text to speech using ElevenLabs voice AI. Use when generating audio from text, creating voiceovers, building voice apps, or synthesizing speech in 90+ languages.

installs
6
GitHub stars
484
Updated
Oct 9
elevenlabs
Official

agents

Build voice AI agents with ElevenLabs. Use when creating voice assistants, customer service bots, interactive voice characters, or any real-time voice conversation experience, and when configuring an agent's tools, workflows, or procedures, including creating, editing, and publishing free-form and structured procedure drafts on an agent branch over the SDKs or REST API.

installs
4
GitHub stars
484
Updated
Oct 9