How to eliminate audio interference from speakers on the microphone while both are in use at the same time using speech recognizing and sythesizing

Question

Hi, I'm creating a real-time voice chatbot. For speech recognition and synthesis I am using Azure Speech. What I do is recognize the voice, then send to an LLM to get a response, and then synthesize the response into audio in real time. My goal is that while the speech synthesis is running, it can continue recognizing at the same time so that when it recognizes audio from the microphone, the speech synthesis ends (synthesizer.stop_speaking()). I currently use one thread for speech recognition and another thread for speech synthesis. The code works well, but the problem I have is that when the voice synthesis is running and at the same time the voice is being recognized by the microphone, sometimes, part of the audio from the voice synthesis is recognized by the microphone. I would like to know if there is any property of speech_recognizer, where I can lower the audio pickup level so that there is no interference or if I should do something different. Thank you very much in advance.

Answer 1

Hello Gerardo,

Thanks for reaching out to us, if you can provide more details about your case, we can provide more information.

To answer your question generally first- Physical Setup and Acoustic Treatments

Microphone Placement:

• Distance: Position the microphone close to the speaker's mouth to capture clean audio. A closer mic can help in reducing background noise and interference.
• Direction: Use directional microphones (e.g., cardioid or supercardioid) that pick up sound primarily from one direction. This helps in minimizing the pickup of sound from speakers.

Speakers Placement:

• Distance: Place speakers away from the microphone to reduce the chance of sound being picked up by the mic. Use directional speakers or speakers with focused sound output if possible.
  • Direction: Ensure speakers are pointed away from the microphone or towards a different part of the room.

Room Acoustics:

• Soundproofing: Use sound-absorbing materials (e.g., acoustic panels, carpets) to reduce echo and reverberation. This helps in minimizing the reflections that the microphone might pick up.
• Isolation: Consider using isolation shields or baffles around the microphone to further reduce interference from speakers.

Electronic and Software Solutions

Echo Cancellation:

• Hardware Solutions: Some audio interfaces and microphones come with built-in echo cancellation features.
• Software Solutions: Use digital signal processing (DSP) algorithms for echo cancellation in your software. Libraries and tools like WebRTC or AEC (Acoustic Echo Cancellation) can be utilized for this purpose. Noise Suppression and Filtering:
  • Noise Suppression: Implement noise suppression algorithms to reduce background noise. Libraries and APIs like Google WebRTC, Microsoft Azure Cognitive Services, or SpeexDSP can be used for real-time noise suppression.
    • High-pass Filtering: Apply high-pass filters to remove low-frequency noise, such as rumble or hums, which might be picked up from the speakers.
    Automatic Gain Control (AGC):
    • AGC adjusts the microphone’s sensitivity based on the incoming audio level. Properly configured AGC can help in balancing the microphone input and reducing unwanted noise from the speakers.
    Beamforming Microphones:
    • Use beamforming microphones that focus on sound coming from a specific direction and reject sound from other directions. These are effective in isolating the speaker's voice from background noise.

Advanced Techniques

Use of Separate Audio Channels:

• Dedicated Channels: If feasible, use separate audio channels for speech recognition and synthesis. This helps in isolating the microphone input from speaker output.
  • Digital Mixing: Employ digital mixers to manage and control audio signals effectively, ensuring that the microphone input is clean and free from interference.

Noise Gate:

• Implement a noise gate in your audio processing setup. A noise gate can help in suppressing audio signals below a certain threshold, which can reduce background noise and interference from speakers.

Adaptive Filtering:

• Use adaptive filtering techniques to dynamically adjust to the changing audio environment. This can help in mitigating interference as the system adapts to the audio characteristics of the room.

I hope this helps, let us know if you need more information.

Regards,

Yutong

-Please kindly accept the answer the question if you feel helpful to support the community, thanks a lot.