bot that will get all audio from call / meeting in MS teams

Question

Hi,

I'm trying to build a bot that will automatically connect to any meeting / call that my user is in and will get the audio from all speakers and will save it for later.

is this even possible?

I've tried to follow the PSI bot example. but I'm getting strange 9999 errors what seems like permissions but I'm not sure.

also if possible is this bot must join the call or any other callback?

I want the audio live and not record

thanks!

Answer 1

Creating a bot that can automatically connect to any meeting or call, capture audio from all speakers, and save it for later is indeed a complex task. It involves several technical and permission-related challenges.

Firstly, capturing live audio from a meeting requires the bot to join the call. This is because the bot needs to be an active participant in the meeting to access the audio streams. The PSI bot example you mentioned is a good starting point, but the 9999 errors you're encountering likely indicate permission issues. These errors can occur if the bot does not have the necessary permissions to join the call or access the audio streams.

To address these issues, you need to ensure that your bot has the following permissions:

1. Application permissions: These are required for the bot to join calls and access audio streams. You need to configure these permissions in the Azure portal.
2. Delegated permissions: These are required if the bot needs to act on behalf of a user. This might be necessary if the bot needs to join meetings scheduled by different users.

Additionally, you need to handle the following aspects:

• Authentication: Ensure that your bot is properly authenticated and authorized to join the meetings.
• Audio processing: Implement the logic to capture and process the audio streams in real-time.

If you want the audio live and not recorded, you will need to implement a real-time audio processing pipeline. This involves capturing the audio streams, processing them as needed, and then saving or streaming the processed audio.

It's also important to consider the legal and privacy implications of capturing and storing audio from meetings. Make sure to comply with relevant regulations and obtain necessary consents from participants.

Document Link-
1.https://learn.microsoft.com/en-us/microsoftteams/platform/bots/calls-and-meetings/real-time-media-concepts
2.https://learn.microsoft.com/en-us/microsoftteams/platform/bots/calls-and-meetings/calls-meetings-bots-overview#calls-and-online-meetings-bots
3.https://github.com/microsoftgraph/microsoft-graph-comms-samples/tree/master/Samples/V1.0Samples/LocalMediaSamples/PolicyRecordingBot

Answer 2

Hey Ellie, Capturing live audio from a Microsoft Teams meeting is possible, but only through an application-hosted bot that uses the Microsoft Graph Calls & Online Meetings APIs.

At a high level:

• Register a Teams calling bot in Azure and enable media streaming. This requires the Calls.AccessMedia.All permission (application type) with admin consent.

The bot must join the meeting as a participant to access the real-time RTP audio packets.

• From there, stream the audio to your processing or storage service, for example, to transcribe or analyze it in real time.

The 9999 errors in the PSI sample typically point to permission or policy configuration issues.

There’s no method to have a bot automatically join every meeting across a tenant; it can only join meetings it’s invited to or authorized for.

You can also build a meeting bot from scratch yourself or use a third-party API. For a step-by-step guide (and open source repo) here's a blog that walks through how to build a MS Teams bot to get transcripts and you could expand to make it work with audio. If you're more interested in a third-party solution there are some blogs that will walk through how to build a meeting bot with one of the third-party APIs.