Live Share Media Synchronization - Teams

發行項
2023/05/24

Video and audio are instrumental parts of the modern world and workplace. We've heard wide ranging feedback that there's more we can do to increase the quality, accessibility, and license protections of watching videos together in meetings.

The Live Share SDK enables robust media synchronization for any HTML <video> and <audio> element with just a few lines of code. By synchronizing media at the player state and transport controls layer, you can individually attribute views and license, while providing the highest possible quality available through your app.

Install

Live Share media is a JavaScript package published on npm, and you can download through npm or yarn. You must also install its peer dependencies, which include @microsoft/live-share, fluid-framework and @fluidframework/azure-client. If you're using Live Share in your tab application, you must also install @microsoft/teams-js version 2.11.0 or later.

Bash

npm install @microsoft/live-share @microsoft/live-share-media fluid-framework @fluidframework/azure-client --save
npm install @microsoft/teams-js --save

To add the latest version of the SDK to your application using Yarn:

Bash

yarn add @microsoft/live-share @microsoft/live-share-media fluid-framework @fluidframework/azure-client
yarn add @microsoft/teams-js

Media sync overview

The Live Share SDK has two primary classes related to media synchronization:

Classes	Description
`LiveMediaSession`	Custom live object designed to coordinate media transport controls and playback state in independent media streams.
MediaPlayerSynchronizer	Synchronizes any object that implements the `IMediaPlayer` interface -- including HTML5 `<video>` and `<audio>` -- using `LiveMediaSession`.

Example:

HTML

<body>
  <video id="player">
    <source src="YOUR_VIDEO_SRC" type="video/mp4" />
  </video>
</body>

JavaScript

import {
  LiveShareClient,
  UserMeetingRole,
  MediaPlayerSynchronizerEvents,
} from "@microsoft/live-share";
import { LiveMediaSession } from "@microsoft/live-share-media";
import { LiveShareHost } from "@microsoft/teams-js";

// Setup the Fluid container
const host = LiveShareHost.create();
const liveShare = new LiveShareClient(host);
const schema = {
  initialObjects: { mediaSession: LiveMediaSession },
};
const { container } = await liveShare.joinContainer(schema);
const { mediaSession } = container.initialObjects;

// Get the player from your document and create synchronizer
const player = document.getElementById("player");
const synchronizer = mediaSession.synchronize(player);

// Listen for groupaction events (optional)
synchronizer.addEventListener(MediaPlayerSynchronizerEvents.groupaction, async (evt) => {
  // See which user made the change (e.g., to display a notification)
  const clientInfo = await synchronizer.mediaSession.getClientInfo(evt.details.clientId);
});

// Define roles you want to allow playback control and start sync
const allowedRoles = [UserMeetingRole.organizer, UserMeetingRole.presenter];
await mediaSession.initialize(allowedRoles);

TypeScript

import {
  LiveShareClient,
  UserMeetingRole,
  MediaPlayerSynchronizerEvents,
} from "@microsoft/live-share";
import { LiveMediaSession, IMediaPlayer, MediaPlayerSynchronizer } from "@microsoft/live-share-media";
import { LiveShareHost } from "@microsoft/teams-js";
import { ContainerSchema } from "fluid-framework";

// Join the Fluid container
const host = LiveShareHost.create();
const liveShare = new LiveShareClient(host);
const schema: ContainerSchema = {
  initialObjects: { mediaSession: LiveMediaSession },
};
const { container } = await liveShare.joinContainer(schema);
// Force casting is necessary because Fluid does not maintain type recognition for `container.initialObjects`.
// Casting here is always safe, as the `initialObjects` is constructed based on the schema you provide to `.joinContainer`.
const mediaSession = container.initialObjects.mediaSession as unknown as LiveMediaSession;

// Get the player from your document and create synchronizer
const player: IMediaPlayer = document.getElementById("player") as HTMLVideoElement;
const synchronizer: MediaPlayerSynchronizer = mediaSession.synchronize(player);

// Listen for groupaction events (optional)
synchronizer.addEventListener(MediaPlayerSynchronizerEvents.groupaction, async (evt) => {
  // See which user made the change (e.g., to display a notification)
  const clientInfo = await synchronizer.mediaSession.getClientInfo(evt.details.clientId);
});

// Define roles you want to allow playback control and start sync
const allowedRoles: UserMeetingRole[] = [UserMeetingRole.organizer, UserMeetingRole.presenter];
await mediaSession.initialize(allowedRoles);

jsx

import { useMediaSynchronizer } from "@microsoft/live-share-react";
import { UserMeetingRole } from "@microsoft/live-share";
import { MediaPlayerSynchronizerEvents } from "@microsoft/live-share-media";
import { useRef, useEffect } from "react";

const ALLOWED_ROLES = [UserMeetingRole.organizer, UserMeetingRole.presenter];

const INITIAL_TRACK = "<YOUR_VIDEO_URL>";

// Define a unique key that distinguishes this `useMediaSynchronizer` from others in your app
const UNIQUE_KEY = "MEDIA-SESSION-ID";

export function VideoPlayer() {
  const videoRef = useRef(null);
  const { play, pause, seekTo, mediaSynchronizer } = useMediaSynchronizer(
    UNIQUE_KEY,
    videoRef,
    INITIAL_TRACK,
    ALLOWED_ROLES
  );

  // Listen for groupaction events (optional)
  useEffect(() => {
    // Listen for player group actions for errors (e.g., play error)
    const onGroupAction = (evt: IMediaPlayerSynchronizerEvent) => {
      // See which user made the change (e.g., to display a notification)
      const clientInfo = await synchronizer.mediaSession.getClientInfo(evt.details.clientId);
    };
    mediaSynchronizer?.addEventListener(
      MediaPlayerSynchronizerEvents.groupaction,
      onGroupAction
    );
    return () => {
      mediaSynchronizer?.removeEventListener(
        MediaPlayerSynchronizerEvents.groupaction,
        onGroupAction
      );
    };
  }, [mediaSynchronizer]);

  return (
    <div>
      <video ref={videoRef} />
      <button onClick={play}>
        Play
      </button>
      <button onClick={pause}>
        Pause
      </button>
      <button onClick={() => {
        seekTo(0);
      }}>
        Start over
      </button>
    </div>
  );
}

The LiveMediaSession automatically listens for changes to the group's playback state. MediaPlayerSynchronizer listens to state changes emitted by LiveMediaSession and applies them to the provided IMediaPlayer object, such as an HTML5 <video> or <audio> element. To avoid playback state changes that a user didn't intentionally initiate, such as a buffer event, we must call transport controls through the synchronizer, rather than directly through the player.

Example:

HTML

<body>
  <video id="player">
    <source src="YOUR_VIDEO_SRC" type="video/mp4" />
  </video>
  <div class="player-controls">
    <button id="play-button">Play</button>
    <button id="pause-button">Pause</button>
    <button id="restart-button">Restart</button>
    <button id="change-track-button">Change track</button>
  </div>
</body>

JavaScript

// ...

document.getElementById("play-button").onclick = async () => {
  // Plays for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  await synchronizer.play();
};

document.getElementById("pause-button").onclick = async () => {
  // Pauses for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  await synchronizer.pause();
};

document.getElementById("restart-button").onclick = async () => {
  // Seeks for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  await synchronizer.seekTo(0);
};

document.getElementById("change-track-button").onclick = () => {
  // Changes the track for all users in the session.
  // If using role verification, this throws an error if the user doesn't have the required role.
  synchronizer.setTrack({
    trackIdentifier: "SOME_OTHER_VIDEO_SRC",
  });
};

注意

While you can use the LiveMediaSession object to synchronize media manually, it's generally recommend to use the MediaPlayerSynchronizer. Depending on the player you use in your app, you might need to create a delegate shim to make your web player's interface match the IMediaPlayer interface.

Suspensions and wait points

If you want to temporarily suspend synchronization for the LiveMediaSession object, you can use suspensions. A MediaSessionCoordinatorSuspension object is local by default, which can be helpful in cases where a user might want to catch up on something they missed, take a break, and so on. If the user ends the suspension, synchronization resumes automatically.

JavaScript

// Suspend the media session coordinator
const suspension = mediaSession.coordinator.beginSuspension();

// End the suspension when ready
suspension.end();

TypeScript

import { MediaSessionCoordinatorSuspension } from "@microsoft/live-share-media";

// Suspend the media session coordinator
const suspension: MediaSessionCoordinatorSuspension = mediaSession.coordinator.beginSuspension();

// End the suspension when ready
suspension.end();

jsx

import { useMediaSynchronizer } from "@microsoft/live-share-react";
import { useRef } from "react";

// Define a unique key that distinguishes this `useMediaSynchronizer` from others in your app
const UNIQUE_KEY = "MEDIA-SESSION-ID";

// Example component
export function VideoPlayer() {
  const videoRef = useRef(null);
  const { suspended, beginSuspension, endSuspension } = useMediaSynchronizer(
    UNIQUE_KEY,
    videoRef,
    "<YOUR_INITIAL_VIDEO_URL>",
  );

  return (
    <div>
      <video ref={videoRef} />
      {!suspended && (
        <button onClick={() => {
          beginSuspension();
        }}>
          Stop following
        </button>
      )}
      {suspended && (
        <button onClick={() => {
          endSuspension();
        }}>
          Sync to presenter
        </button>
      )}
    </div>
  );
}

When beginning a suspension, you can also include an optional CoordinationWaitPoint parameter, which allows users to define the timestamps in which a suspension should occur for all users. Synchronization doesn't resume until all users end the suspension for that wait point.

Here are a few scenarios where wait points are especially useful:

Adding a quiz or survey at certain points in the video.
Waiting for everyone to suitably load a video before it starts or while buffering.
Allow a presenter to choose points in the video for group discussion.

JavaScript

// Suspend the media session coordinator
const waitPoint = {
  position: 0,
  reason: "ReadyUp", // Optional.
};
const suspension = mediaSession.coordinator.beginSuspension(waitPoint);
// End the suspension when the user readies up
document.getElementById("ready-up-button").onclick = () => {
  // Sync resumes when everyone ends suspension
  suspension.end();
};

TypeScript

import { MediaSessionCoordinatorSuspension, CoordinationWaitPoint } from "@microsoft/live-share-media";

// Suspend the media session coordinator
const waitPoint: CoordinationWaitPoint = {
  position: 0,
  reason: "ReadyUp", // Optional.
};
const suspension = mediaSession.coordinator.beginSuspension(waitPoint);

// End the suspension when the user readies up
document.getElementById("ready-up-button")!.onclick = () => {
  // Sync resumes when everyone ends suspension
  suspension.end();
};

jsx

import { useMediaSynchronizer } from "@microsoft/live-share-react";
import { useRef } from "react";

// Define a unique key that distinguishes this `useMediaSynchronizer` from others in your app
const UNIQUE_KEY = "MEDIA-SESSION-ID";

// Example component
export function VideoPlayer() {
    const videoRef = useRef(null);
    const { suspended, beginSuspension, endSuspension } = useMediaSynchronizer(
        UNIQUE_KEY,
        videoRef,
        "<YOUR_INITIAL_VIDEO_URL>",
    );
    
    return (
      <div>
          <video ref={videoRef} />
          {!suspended && (
            <button onClick={() => {
                const waitPoint = {
                  position: 0,
                  reason: "ReadyUp", // Optional.
                };
                beginSuspension(waitPoint);
            }}>
                Wait until ready
            </button>
          )}
          {suspended && (
            <button onClick={() => {
              endSuspension();
            }}>
                Ready up
            </button>
          )}
      </div>
    );
}

Audio ducking

The Live Share SDK supports intelligent audio ducking. You can use the feature in your application by adding the following to your code:

JavaScript

import { meeting } from "@microsoft/teams-js";

// ... set up MediaPlayerSynchronizer

// Register speaking state change handler through Microsoft Teams JavaScript client library (TeamsJS)
let volumeTimer;
meeting.registerSpeakingStateChangeHandler((speakingState) => {
  if (speakingState.isSpeakingDetected && !volumeTimer) {
    // If someone in the meeting starts speaking, periodically
    // lower the volume using your MediaPlayerSynchronizer's
    // VolumeLimiter.
    synchronizer.volumeLimiter?.lowerVolume();
    volumeTimer = setInterval(() => {
      synchronizer.volumeLimiter?.lowerVolume();
    }, 250);
  } else if (volumeTimer) {
    // If everyone in the meeting stops speaking and the
    // interval timer is active, clear the interval.
    clearInterval(volumeTimer);
    volumeTimer = undefined;
  }
});

TypeScript

import { meeting } from "@microsoft/teams-js";

// ... set up MediaPlayerSynchronizer

// Register speaking state change handler through TeamsJS library
let volumeTimer: NodeJS.Timeout | undefined;
meeting.registerSpeakingStateChangeHandler((speakingState: meeting.ISpeakingState) => {
  if (speakingState.isSpeakingDetected && !volumeTimer) {
    // If someone in the meeting starts speaking, periodically
    // lower the volume using your MediaPlayerSynchronizer's
    // VolumeLimiter.
    synchronizer.volumeLimiter?.lowerVolume();
    volumeTimer = setInterval(() => {
      synchronizer.volumeLimiter?.lowerVolume();
    }, 250);
  } else if (volumeTimer) {
    // If everyone in the meeting stops speaking and the
    // interval timer is active, clear the interval.
    clearInterval(volumeTimer);
    volumeTimer = undefined;
  }
});

jsx

import { useMediaSynchronizer } from "@microsoft/live-share-react";
import { meeting } from "@microsoft/teams-js";
import { useRef, useEffect } from "react";

// Define a unique key that distinguishes this `useMediaSynchronizer` from others in your app
const UNIQUE_KEY = "MEDIA-SESSION-ID";

// Example component
export function VideoPlayer() {
    const videoRef = useRef(null);
    const { synchronizer } = useMediaSynchronizer(
        UNIQUE_KEY,
        videoRef,
        "<YOUR_INITIAL_VIDEO_URL>",
    );

    const enableSmartSound = () => {
        let volumeTimer;
        meeting.registerSpeakingStateChangeHandler((speakingState) => {
            if (speakingState.isSpeakingDetected && !volumeTimer) {
                // If someone in the meeting starts speaking, periodically
                // lower the volume using your MediaPlayerSynchronizer's
                // VolumeLimiter.
                synchronizer.volumeLimiter?.lowerVolume();
                volumeTimer = setInterval(() => {
                    synchronizer.volumeLimiter?.lowerVolume();
                }, 250);
            } else if (volumeTimer) {
                // If everyone in the meeting stops speaking and the
                // interval timer is active, clear the interval.
                clearInterval(volumeTimer);
                volumeTimer = undefined;
            }
        });
    }

    return (
        <div>
            <video ref={videoRef} />
            <button onClick={enableSmartSound}>
                Enable smart sound
            </button>
        </div>
      );
}

Additionally, add the following RSC permissions into your app manifest:

JSON

{
  // ...rest of your manifest here
  "authorization": {
    "permissions": {
      "resourceSpecific": [
        // ...other permissions here
        {
          "name": "OnlineMeetingIncomingAudio.Detect.Chat",
          "type": "Delegated"
        },
        {
          "name": "OnlineMeetingIncomingAudio.Detect.Group",
          "type": "Delegated"
        }
      ]
    }
  }
}

注意

The registerSpeakingStateChangeHandler API used for audio ducking only works on Microsoft Teams desktop and in scheduled and meet now meeting types.

Code samples

Sample name	Description	JavaScript	TypeScript
React video	Basic example showing how the LiveMediaSession object works with HTML5 video.	View	NA
React media template	Enable all connected clients to watch videos together, build a shared playlist, transfer whom is in control, and annotate over the video.	View	View

Next step

Live Share canvas

其他資源

文件

Live Share overview - Teams

Learn about Live Share SDK, Live share core, Live share media, and Live share canva, apps with Live Share, Live Share collaborative contexts, and React integration.
Live Share FAQ - Teams

Learn more about Live Share SDK frequently asked questions related to Azure Fluid Relay service, data stored in Live Share, fluid containers, and government clouds.
Collaborate with Live Share Canvas - Teams

In this module, learn more about Live Share canvas, an extension enabling inking, laser pointers, and cursors for meeting apps.
Live Share quick start - Teams

Learn how to quickly build and run the Dice Roller sample using Microsoft Live Share SDK with Microsoft Visual Studio Code, Node.js, and Git.
Live Share getting started - Teams

Learn about Live Share SDK capabilities, data structure, join a session, RSC permissions, role verification for live Share and fluid distribution data structures.
Apps for Teams Meeting Stage - Teams

Learn how to build apps for Teams meeting stage, share to stage APIs, and generate a deep link to share content to stage in meetings.
Meeting Side Panel - Teams

Learn how to generate meeting side panel to enhance Microsoft Teams meetings experience and interact in the side panel to add and share agendas.
Create Tab Apps for Calling - Teams

Learn to build a tab for initiating one-to-one calls from within Teams

訓練

模組

結合外部應用程式以取得 Microsoft 365 的其它自訂產品 - Training

瞭解如何在 Microsoft Teams 中為大批觀眾建立專業且吸引人的簡報。

認證

Microsoft 365 Certified: Collaboration Communications Systems Engineer Associate - Certifications

示範如何設定、部署、監視及管理 Microsoft Teams 電話、會議和認證裝置的技能。

共用方式為

Install

Media sync overview

Suspensions and wait points

Audio ducking

Code samples

Next step

See also

其他資源

共用方式為

Live Share media capabilities

Install

Media sync overview

Suspensions and wait points

Audio ducking

Code samples

Next step

See also

其他資源