How to reconnect Azure Speech SDK ConversationTranscriber session in Python without silence or end of stream?

Question

Environment:

SDK: Azure Speech SDK for Python

• Version: Latest

Region: eastus

• OS: ubuntu
• Audio Input: Microphone (WebM(browser) → WAV(backend python) conversion via GStreamer)
• GStreamer core version: 1.20.3
• azure-cognitiveservices-speech version: 1.46.0

Scenario: Real-time transcription using ConversationTranscriber

---

Problem Description

When using the ConversationTranscriber for real-time speech recognition, the transcription session automatically stops when the SDK detects silence or when the input audio stream ends.

I need to manually control when the transcriber stops or restarts (for example, when a user clicks a “Stop” or “Restart” button in the UI). However, currently, the SDK seems to end the session automatically after a period of silence or an end-of-stream signal.

---

Questions

1. Manual control: How can use my transcription session from Python code (e.g., using a function call), instead of letting the SDK stop automatically due to silence or end of stream?

Reconnection / Reuse: After manually stopping a session, can I restart or reconnect using the same ConversationTranscriber or SpeechConfig instance? Or do I need to create a new ConversationTranscriber object each time I start a new transcription session?

1. Session management best practices: What’s the recommended pattern for handling multiple manual transcription sessions within the same Python process?

Silence gap handling: Is there any SDK configuration or parameter to prevent auto-ending due to silence or to extend the silence timeout (e.g., setting a longer InitialSilenceTimeout or disabling automatic end-of-stream detection)?

Accuracy and speaker identification: What are the recommended best practices to improve transcription accuracy and speaker diarization quality for conversation transcription? For example:

Preferred audio format (WAV, PCM, sampling rate, bit depth)

  Recommended microphone setup (mono/stereo, close-talk array)

  
     Relevant `SpeechServiceConnection_*` properties to adjust performance

     
        Any settings in the Azure Speech resource (e.g., custom models or channel setup) that help with speaker separation
```---
Expected Behavior

I should be able to:

Prevent the SDK from auto-ending due to short silence gaps.

Restart or continue a session without needing to fully recreate the transcriber and configuration objects.

Actual Behavior

- The session currently ends automatically after silence or audio stream termination.

Answer 1

Hi Nagalakshmi J

By default, the Azure Speech SDK ends a transcription session when:

• It detects prolonged silence.
• It receives an end-of-stream signal from the audio input.

This behavior is built-in for efficiency but can be overridden.

How to prevent auto-ending due to silence

You can adjust or disable silence detection using SDK properties:

• Increase silence timeout: speechconfig.setproperty(    speechsdk.PropertyId.SpeechServiceConnection_InitialSilenceTimeoutMs,    "60000"  # e.g., 60 seconds) Reference -https://learn.microsoft.com/en-us/dotnet/api/microsoft.cognitiveservices.speech.propertyid?view=azure-dotnet
  • Disable end-of-stream auto-stop: Currently, there’s no direct “disable” flag, but you can keep the audio stream open and avoid calling push_stream.close() until you want to stop.

Manual session control

Use these methods:

• Start transcription: await conversationtranscriber.starttranscribing_async()
• Stop transcription manually: await conversationtranscriber.stoptranscribing_async()

Attach event handlers (session_stopped, canceled) to monitor state.

Reference - https://github.com/Azure-Samples/cognitive-services-speech-sdk/blob/master/samples/python/console/transcription_sample.py

Reconnection / Reuse

• Best practice: Create a new ConversationTranscriber instance for each session.\ Reusing the same object after stop_transcribing_async() is not recommended because internal state may not reset cleanly.
• You can reuse the same SpeechConfig and AudioConfig objects across sessions.

Session management pattern

For multiple sessions in one process:

• Keep a single SpeechConfig (with subscription key and region).
• For each new session:
  • Create a new ConversationTranscriber.
    • Call start_transcribing_async() and stop_transcribing_async() as needed.
• For better accuracy and speaker diarization:
  • Use uncompressed PCM audio (16 kHz, 16-bit mono).
  • Configure speech_config.set_property_by_name("maxSpeakerCount", "8") if needed.
  • Consider custom models for speaker separation.

I am facing some issue testing the sample code,

import azure.cognitiveservices.speech as speechsdk

# 1. Configure Speech Service
speech_key = "<YOUR_SUBSCRIPTION_KEY>"
service_region = "<YOUR_REGION>"

speech_config = speechsdk.SpeechConfig(subscription=speech_key, region=service_region)

# Configure silence timeout (e.g., 60 seconds)
speech_config.set_property(
    speechsdk.PropertyId.SpeechServiceConnection_InitialSilenceTimeoutMs,
    "60000"  # 60,000 ms = 60 seconds
)

# Optional: Configure max speaker count for diarization
speech_config.set_property_by_name("maxSpeakerCount", "8")

# 2. Create AudioConfig (microphone or custom stream)
audio_config = speechsdk.audio.AudioConfig(use_default_microphone=True)

# Function to create a new ConversationTranscriber
def create_transcriber():
    return speechsdk.conversation.ConversationTranscriber(
        speech_config=speech_config,
        audio_config=audio_config
    )

# 3. Start and Stop Transcription
async def run_transcription():
    # Create a new transcriber instance
    transcriber = create_transcriber()

    # Attach event handlers
    transcriber.transcribed.connect(lambda evt: print(f"TRANSCRIBED: {evt.result.text}"))
    transcriber.session_stopped.connect(lambda evt: print("Session stopped"))
    transcriber.canceled.connect(lambda evt: print(f"Canceled: {evt.reason}"))

    # Start transcription
    print("Starting transcription...")
    await transcriber.start_transcribing_async()

    # Simulate some work (e.g., wait for audio input)
    await asyncio.sleep(30)  # Replace with your logic

    # Stop transcription manually
    print("Stopping transcription...")
    await transcriber.stop_transcribing_async()

    # Recreate transcriber for next session
    print("Reconnecting...")
    transcriber = create_transcriber()
    await transcriber.start_transcribing_async()
    # Repeat as needed

# Run the async function
import asyncio
asyncio.run(run_transcription())

please refer this sample code for modification

Hope it helps.

Thank you