calesthio/openmontage

azure-speech-to-text

Transcribe audio to text using Azure AI Speech (Fast Transcription REST API).

查看源码

使用方法

  1. 复制安装命令,在自己的智能体环境安装前先检查源码包。
  2. 从原始文档核对所需工具、凭据和支持的宿主。
  3. 说明输入材料与期望交付物,并对照来源检查输出。
npx skills add https://github.com/calesthio/openmontage --skill azure-speech-to-text

开始前需了解

  • 安装不包含模型订阅、API 额度或访问文件的权限。
  • 仓库星数和安装量反映活跃情况,不等于独立质量或安全测试。
仓库原始内容

按源仓库内容呈现,保留标题、案例、代码、表格、链接以及原文引用的演示图片。

Azure AI Speech — Speech-to-Text

Transcribe audio to text with Azure Fast Transcription — synchronous, word-level timestamps, speaker diarization, and multi-language identification. In OpenMontage this is exposed through the azure_stt tool (capability=analysis, provider=azure). It is an optional cloud STT provider — when AZURE_SPEECH_KEY is configured, prefer it for cloud transcription. The local transcriber tool (faster-whisper) remains the default offline path and the fallback when Azure is unavailable.

Docs: Fast Transcription · Speech service overview

Why Fast Transcription (not Batch)

Azure exposes three STT surfaces. OpenMontage uses Fast Transcription because the pipeline transcribes local audio files:

SurfaceInputLatencyNeeds
Fast Transcription (used here)local file, multipart POSTsynchronous, sub-real-timekey + region
Batch Transcriptionaudio at a URL (Blob + SAS)async job + pollingBlob storage plumbing
Speech SDK (spx)mic / stream / filestreamingnative azure-cognitiveservices-speech package

Fast Transcription needs no Blob storage, no SAS URLs, and no native SDK — just requests and the two env vars.

Setup

Create a Speech resource in the Azure portal; copy the key and region from its Keys and Endpoint page.

bash
export AZURE_SPEECH_KEY=your_speech_resource_key
export AZURE_SPEECH_REGION=eastus          # your resource's region
# export AZURE_SPEECH_ENDPOINT=https://...  # optional: overrides region

azure_stt reports AVAILABLE once AZURE_SPEECH_KEY plus either AZURE_SPEECH_REGION or AZURE_SPEECH_ENDPOINT are set.

Using it in a pipeline

Prefer azure_stt over transcriber unless the run must be offline. Its output matches the transcriber schema exactly, so it is a drop-in for subtitle_gen and any stage that consumes a transcript.

python
from tools.tool_registry import registry
registry.discover()
stt = registry._tools["azure_stt"]

result = stt.execute({
    "input_path": "projects/my-video/assets/audio/narration.mp3",
    # "language": "en",          # ISO 639-1 or BCP-47 ("en-US"); omit for auto-ID
    # "diarize": True,           # speaker labels, no HuggingFace token needed
    # "max_speakers": 4,
    "output_dir": "projects/my-video/artifacts",
})
if result.success:
    segs = result.data["segments"]          # [{id,start,end,text,words:[...]}]
    words = result.data["word_timestamps"]  # flat [{word,start,end,probability}]

If azure_stt is unavailable (no key) or errors, fall back to transcriber (local whisper) — its execute signature and output are identical.

Parameters that matter

  • `language` — pass an ISO code ("en") or a full locale ("en-US"). Pin it

when you know the language; it is faster and more accurate than auto-ID.

  • `candidate_locales` — when language is omitted, Azure runs language

identification across this shortlist. Narrow it to the languages you actually expect; a huge list slows detection and invites misclassification.

  • `diarize` / `max_speakers` — enable for multi-speaker audio (interviews,

podcasts). Set max_speakers to the real upper bound.

  • `profanity_filter` — None | Masked (default) | Removed | Tags.

Response shape (mapped to the transcriber schema)

The raw Azure response (phrases[] with offsetMilliseconds / words[]) is converted to seconds and the OpenMontage transcript schema:

json
{
  "segments": [
    {"id": 0, "start": 0.0, "end": 2.4, "text": "Hello world",
     "speaker": 1,
     "words": [{"word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.98}]}
  ],
  "word_timestamps": [{"word": "Hello", "start": 0.0, "end": 0.5, "probability": 0.98}],
  "language": "en-US",
  "duration_seconds": 2.4,
  "provider": "azure"
}

Note: Fast Transcription has no per-word confidence, so each word carries the phrase confidence in probability.

Limits & tips

  • Single file up to ~2 hours / a few hundred MB per request. For longer or bulk

jobs, use Azure Batch Transcription instead.

  • Send clean audio (16 kHz+ mono is plenty). Transcode video to audio first if

you only need speech — smaller upload, same result.

  • Verify timing: word timestamps drive subtitle cues in subtitle_gen. Spot-check

the first and last cues against the source audio.

按任务相关推荐

来自同一仓库

更多 Skills

全部 Skills
calesthio
社区

hyperframes-cli

HyperFrames CLI dev loop. Use when running npx hyperframes init, add, catalog, capture, lint, validate, inspect, layout, snapshot, preview, play, render, publish, lambda, doctor, browser, info, upgrade, skills, compositions, docs, benchmark, telemetry, transcribe, tts, or remove-background, or when troubleshooting the HyperFrames build/render environment. Entry point for AWS Lambda cloud rendering (hyperframes lambda deploy / render / progress / destroy / policies).

1小时安装量
2
GitHub Stars
7.8万
最近更新
11月3日
calesthio
社区

pose-library-design

Design reusable 2D character pose libraries, action cycles, and expression states for data-driven animation.

1小时安装量
2
GitHub Stars
7.8万
最近更新
11月3日
calesthio
社区

create-video

Create videos from a text prompt using HeyGen's Video Agent. Use when: (1) Creating a video from a description or idea, (2) Generating explainer, demo, or marketing videos from a prompt, (3) Making a video without specifying exact avatars, voices, or scenes, (4) Quick video prototyping or drafts, (5) One-shot prompt-to-video generation, (6) User says "make me a video" or "create a video about X".

1小时安装量
2
GitHub Stars
7.6万
最近更新
11月3日
calesthio
社区

ffmpeg

Video and audio processing with FFmpeg. Use for format conversion, resizing, compression, audio extraction, and preparing assets for Remotion. Triggers include converting GIF to MP4, resizing video, extracting audio, compressing files, or any media transformation task.

1小时安装量
2
GitHub Stars
7.6万
最近更新
11月3日