Azure AI Video Indexer insights

  • Article

Important

Due to the Azure Media Services retirement announcement, Azure AI Video Indexer announces Azure AI Video Indexer features adjustments. See Changes related to Azure Media Service (AMS) retirement to understand what this means for your Azure AI Video Indexer account. See the Preparing for AMS retirement: VI update and migration guide.

When a video is indexed, Azure AI Video Indexer analyzes the video and audio content by running 30+ AI models, generating rich insights. Insights contain an aggregated view of the data: transcripts, optical character recognition elements (OCRs), face, topics, emotions, etc. Once the video is indexed and analyzed, Azure AI Video Indexer produces a JSON content that contains details of the video insights. For example, each insight type includes instances of time ranges that show when the insight appears in the video.

Read details about the following insights here:

For information about features and other insights, see:

Once you set up an Azure AI Video Indexer account (see account types) and upload a video, you can view insights as described below.

Get the insights using the website

To visually examine the video's insights, press the Play button on the video on the Azure AI Video Indexer website.

Screenshot of the Insights tab in Azure AI Video Indexer.

To get insights produced on the website or the Azure portal:

  1. Browse to the Azure AI Video Indexer website and sign in.

  2. Find a video whose output you want to examine.

  3. Press Play.

  4. Choose the Insights tab.

  5. Select which insights you want to view (under the View drop-down, on the right-top corner).

  6. Go to the Timeline tab to see timestamped transcript lines.

  7. Select Download > Insights (JSON) to get the insights output file.

  8. If you want to download artifacts, beware of the following:

    Warning

    We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

Get insights produced by the API

When indexing with an API and the response status is OK, you get a detailed JSON output as the response content. When calling the Get Video Index API, we recommend passing &includeSummarizedInsights=false.

The JSON output produced by the website or API contains Insights and SummarizedInsights elements. We highly recommend using Insights and not using SummarizedInsights (which is present for backward compatibility).

This API returns a URL only with a link to the specific resource type you request. An additional GET request must be made to this URL for the specific artifact. The file types for each artifact type vary depending on the artifact.

Warning

We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

Examine the Azure AI Video Indexer output

For more information, see Examine the Azure AI Video Indexer output.