Azure Video Indexer insights

When a video is indexed, Azure Video Indexer analyzes the video and audio content by running 30+ AI models, generating rich insights. Insights contain an aggregated view of the data: transcripts, optical character recognition elements (OCRs), face, topics, emotions, etc. Once the video is indexed and analyzed, Azure Video Indexer produces a JSON content that contains details of the video insights. For example, each insight type includes instances of time ranges that show when the insight appears in the video.

Here some common insights:

Insight Description
Audio effects For more information, see Audio effects detection.
Scenes, shots, and keyframes Selects the frame(s) that best represent each shot. Keyframes are the representative frames selected from the entire video based on aesthetic properties (for example, contrast and stableness). Scenes, shots, and keyframes are merged into one insight for easier consumption and navigation. When you select the desired scene you can see what shots and keyframes it consists of. For more information, see Scenes, shots, and keyframes.
Emotions Identifies emotions based on speech and audio cues.
Faces For more information, see Faces detection.
Textual logo detection Matches a specific predefined text using Azure Video Indexer OCR. For example, if a user created a textual logo: "Microsoft", different appearances of the word Microsoft will be detected as the "Microsoft" logo. For more information, see Detect textual logo.
Keywords For more information, see Keywords extraction.
Labels For more information, see Labels identification
Named entities For more information, see Named entities.
People For more information, see Observed people tracking & matched faces.
Topics For more information, see Topics inference.
OCR For more information, see OCR.
Sentiments Sentiments are aggregated by their sentimentType field (Positive, Neutral, or Negative).
Speakers Maps and understands which speaker spoke which words and when. Sixteen speakers can be detected in a single audio-file.
Transcript For more information, see Transcription, translation, language.

For information about features and other insights, see Azure Video Indexer insights.

Once you set up an Azure Video Indexer account (see account types) and upload a video, you can view insights as described below.

Get the insights using the website

To visually examine the video's insights, press the Play button on the video on the Azure Video Indexer website.

Screenshot of the Insights tab in Azure Video Indexer.

To get insights produced on the website or the Azure portal:

  1. Browse to the Azure Video Indexer website and sign in.

  2. Find a video whose output you want to examine.

  3. Press Play.

  4. Choose the Insights tab.

  5. Select which insights you want to view (under the View drop-down, on the right-top corner).

  6. Go to the Timeline tab to see timestamped transcript lines.

  7. Select Download > Insights (JSON) to get the insights output file.

  8. If you want to download artifacts, beware of the following:

    Warning

    We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

Get insights produced by the API

When indexing with an API and the response status is OK, you get a detailed JSON output as the response content. When calling the Get Video Index API, we recommend passing &includeSummarizedInsights=false.

The JSON output produced by the website or API contains Insights and SummarizedInsights elements. We highly recommend using Insights and not using SummarizedInsights (which is present for backward compatibility).

This API returns a URL only with a link to the specific resource type you request. An additional GET request must be made to this URL for the specific artifact. The file types for each artifact type vary depending on the artifact.

Warning

We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

Examine the Azure Video Indexer output

For more information, see Examine the Azure Video Indexer output.

Next steps

View and edit video insights.