For the first time, real-time transcription goes multilingual

When we introduced Post-Stream Refinement earlier this year, it closed the oldest gap in real-time speech: you could finally get instant streaming results and a highly accurate final transcript, with no latency penalty. But it kept one hard requirement — you had to tell the service, up front, which single language to expect. Real-world speech does not work that way. People code-switch mid-sentence, product and brand names cross languages, and a global app serves users who simply speak differently from one session to the next.

Today we remove that requirement. Multilingual Post-Stream Refinement enters public preview for Azure AI Speech in Microsoft Foundry, and for the first time ever a single real-time stream can transcribe multiple languages in one session — the spoken language is detected automatically, no locale is declared in advance, and the final transcript is refined for accuracy. Everything you already know about Post-Stream Refinement still applies; what changes is that the refinement pass itself is now multilingual. 📖 Read the Documentation

What's New in This Release

If you have already used Post-Stream Refinement, here is exactly what changes with the multilingual preview — and what stays the same:

What's new: the same real-time contract, now multilingual. One multilingual model runs the refinement pass, adding automatic multi-language detection, code-switching within a single utterance, open-range setup with no candidate list, and included diarization — while first-token streaming latency stays unchanged.

Quality Impact

In internal testing and partner evaluations across Tier-1 locales, multilingual Post-Stream Refinement reduced word error rate (WER) by approximately 10% relative on average, with double-digit relative reductions on the hardest cases — long utterances, proper nouns, and multilingual or code-switched speech. Partial-result latency is unchanged; only the final transcript is refined.

Gains are relative reductions versus the standard real-time model and vary by language, acoustic conditions, and content type. The refined final result may add a small amount of latency to the final segment; partial results are unaffected.

Measured quality gains (relative improvement vs. standard real-time transcription): about 10% relative reduction in average word error rate across Tier-1 locales; double-digit relative reduction on long utterances; double-digit relative reduction on proper nouns and named entities; no change to partial-result (first-token) latency. Relative reductions in word error rate; public preview, exact figures vary by locale and audio.

Supported Languages and Regions

The public preview supports 15 Tier-1 locales. Because language is detected automatically, a single stream can contain any mix of them:

Supported languages: 25 auto-detected languages spanning 29 market locales — any mix of which can appear in a single real-time stream without declaring a locale up front.

 

Available in these Azure regions:

Availability: multilingual Post-Stream Refinement is live in six Azure regions at public preview — East US, West US, North Europe, Central India, Southeast Asia, and Japan East.

Real-World Impact

Preview customers across industries — including travel, consumer electronics, automotive, aviation, and media — have reported positive gains in transcription quality. Customers testing multilingual and domain-specific audio have observed the clearest improvements on the hardest content: proper nouns, code-switching, and long-form speech. Several are actively validating the feature on their own audio ahead of general availability.

Get Started

Enabling multilingual Post-Stream Refinement is a small configuration change on your existing SpeechConfig. You will need:

• Speech SDK 1.50 or later. Earlier versions do not support the multilingual path.
• A Speech resource in one of the supported regions listed above.
• Auto-detect language configuration (open range) so the service identifies the language from the audio — no candidate list required.

Set the post-processing option to PostRefinement and pass an open-range AutoDetectSourceLanguageConfig when you create the recognizer. Here is a complete, copy-paste Python example, including the optional end-of-utterance detection line:

import azure.cognitiveservices.speech as speechsdk

speech_config = speechsdk.SpeechConfig(
    subscription="YourSpeechKey",
    region="YourSpeechRegion")

# 1) Refine the final transcript (Post-Stream Refinement)
speech_config.set_property(
    speechsdk.PropertyId.SpeechServiceResponse_PostProcessingOption,
    "PostRefinement")

# 2) Multilingual auto-detect - no candidate language list needed
auto_detect_config = speechsdk.languageconfig.AutoDetectSourceLanguageConfig()

audio_config = speechsdk.AudioConfig(use_default_microphone=True)

recognizer = speechsdk.SpeechRecognizer(
    speech_config=speech_config,
    auto_detect_source_language_config=auto_detect_config,
    audio_config=audio_config)

💡 Tip: Refinement matters most for applications that store or process the final transcript — meeting notes, call analytics, compliance archives, AI summarization. If you only use partial results for a live display and discard them, your real-time UX (already fast) is unchanged, while any final transcript you keep improves.

Try Multilingual Post-Stream Refinement Today

Turn on higher-accuracy, language-aware transcription in your Azure AI Speech applications with a single configuration change. Available now in public preview in Microsoft Foundry.

📖 Read the Documentation

We would love your feedback. Try Post-Stream Refinement in your applications and tell us how it improves your transcription quality.