Azure Video Indexer frequently asked questions

This article answers frequently asked questions about Azure Video Indexer.

General questions

What is Azure Video Indexer?

Azure Video Indexer is an artificial intelligence service that is part of Microsoft Azure Media Services. Azure Video Indexer provides an orchestration of multiple machine learning models that enable you to easily extract deep insight from a video. To provide advanced and accurate insights, Azure Video Indexer makes use of multiple channels of the video: audio, speech, and visual. Azure Video Indexer’s insights may be used in many ways, like improving content discoverability and accessibility, creating new monetization opportunities, or building new experiences that use the insights. Azure Video Indexer provides a web-based interface for testing, configuration, and customization of models in your account. Developers can use a REST-based API to integrate Azure Video Indexer into production system.

What can I do with Azure Video Indexer?

Some of the operations that Azure Video Indexer can perform on media files include:

  • Identifying and extracting speech and identify speakers.
  • Identifying and extracting on-screen text in a video.
  • Detecting objects in a video file.
  • Identify brands (for example: Microsoft) from audio tracks and on-screen text in a video.
  • Detecting and recognizing faces from a database of celebrities and a user-defined database of faces.
  • Extracting topics discussed but not necessarily mentioned in audio and video content.
  • Creating closed captions or subtitles from the audio track.

For more information and more Azure Video Indexer features, see Overview.

How do I get started with Azure Video Indexer?

Azure Video Indexer includes a trial offering that provides you with 600 minutes in the web-based interface and 2,400 minutes via the API. You can login to the Azure Video Indexer web-based interface and try it for yourself using any web identity and without having to set up an Azure Subscription. Follow this easy introduction lab to get better idea of how to use Azure Video Indexer.

To index videos and audio flies at scale, you can connect Azure Video Indexer to a paid Microsoft Azure subscription. You can find more information on pricing on the pricing page.

You can find more information on getting started in Get started.

Do I need coding skills to use Azure Video Indexer?

You can use the Azure Video Indexer web-based interface to evaluate, configure, and manage your account with no coding required. When you are ready to develop more complex applications, you can use the Azure Video Indexer API to integrate Azure Video Indexer into your own applications, web sites, or custom workflows using serverless technologies like Azure Logic Apps or Azure Functions.

What are the Limited Access features of Azure Video Indexer and how to register?

Face Identify and Celebrity Recognition features of Azure Video Indexer are Limited Access features that require registration.

Find the registration form and more information about Limited Access in this article.

Also, see the following: the announcement blog post and investment and safeguard for facial recognition blog post.

Do I need machine learning skills to use Azure Video Indexer?

No, Azure Video Indexer provides the integration of multiple machine learning models into one pipeline. Indexing a video or audio file via Azure Video Indexer retrieves a full set of insights extracted on one shared timeline without any machine learning skills or knowledge on algorithms needed on the customer's part.

What media formats does Azure Video Indexer support?

Azure Video Indexer supports most common media formats. Refer to the Azure Media Encoder standard formats list for more details.

How do I upload a media file into Azure Video Indexer and what are the limitations?

In the Azure Video Indexer web-based portal, you can upload a media file using the file upload dialog or by pointing to a URL that directly hosts the source file (see example). Any URL that hosts the media content using an iFrame or embed code will not work (see example).

For more information, please see this how-to guide.

How many files can I stitch and render in a project?

In Azure Video Indexer you can create a project and add multiple files to stitch and render as a new file. The number of source files is set to 10 in the website and 100 in the API. This is a limit set by the Azure Media Services Media Encoder Standard API where we have a dependency on.

How long does it take Azure Video Indexer to extract insights from media?

The amount of time it takes to index a video or audio file, both using the Azure Video Indexer API and the Azure Video Indexer web-based interface, depends on multiple parameters such as the file length and quality, the number of insights found in the file, the number of reserved units available, and whether the streaming endpoint is enabled or not. We recommend that you run a few test files with your own content and take an average to get a better idea.

Can I create customized workflows to automate processes with Azure Video Indexer?

Yes, you can integrate Azure Video Indexer into serverless technologies like Logic Apps, Flow, and Azure Functions. You can find more details on the Logic App and Flow connectors for Azure Video Indexer here. You can see some automation projects done by partners in the Azure Video Indexer Samples repo.

In which Azure regions is Azure Video Indexer available?

You can see which Azure regions Azure Video Indexer is available on the regions page.

Can I customize Azure Video Indexer models for my specific use case?

Yes. In Azure Video Indexer you can customize some of the available models to better fit your needs.

For example, our Person model supports out-of-the-box 1,000,000 faces of celebrity recognition, but you can also train it to recognize other faces which are not in that database.

For details, see articles about customizing Person, Brands, and Language models.

Can I edit the videos in my library?

Yes. Press the edit video button from the library display or the open in editor button from the player display to get to the Projects tab. You can create a new project and add more videos from your library to edit them together, once you are done you can render your video and download.

If you want to get insights on your new video, index it with Azure Video Indexer and it will appear in your library with its insights.

Can I index multiple audio streams or channels?

If there are multiple audio streams, Azure Video Indexer takes the first one it encounters and will process only this stream. In any audio stream Azure Video Indexer processes, it takes the different channels (if present) and processes them together as mono. For streams/channels manipulation you can use ffmpeg commands on the file before indexing it.

Can a storage account connected to the Media Services account be behind a firewall?

Your paid Azure Video Indexer account uses the specified Media Services account that is connected to a storage account. Currently, the use of a connected storage account that is behind a firewall and not publicly accessible is not supported.

What is the SLA for Azure Video Indexer?

Azure Media Service’s SLA covers Azure Video Indexer and can be found on the SLA page. The SLA only applies to Azure Video Indexer paid accounts and does not apply to the trial.

Can’t find my new ARM account in the account list on the Azure Video Indexer website.

For detailed description on how to switch to the correct domain, see Switch between tenants.

Privacy Questions

Are video and audio files indexed by Azure Video Indexer stored?

Yes, unless you delete the file from Azure Video Indexer, either using the Azure Video Indexer website or API, your video and audio files are stored. For the trial, the video and audio files that you index are stored in the Azure region East US. Otherwise, your video and audio files are stored in the storage account of your Azure subscription.

Where does Azure Video Indexer store customer data?

By default, your data is replicated to the paired region into which you deployed the Azure Video Indexer instance. For region with data residency requirements, your data is always kept within the same region. For more related information, check out other Privacy Questions in this section.

Can I delete my files that are stored in the Azure Video Indexer website?

Yes, you can always delete your video and audio files as well as any metadata and insights extracted from them by Azure Video Indexer. Once you delete a file from Azure Video Indexer, the file and its metadata and insights are permanently removed from Azure Video Indexer. However, if you have implemented your own backup solution in Azure storage, the file remains in your Azure storage.

Can I control user access to my Azure Video Indexer account?

Yes, only account admins can invite and uninvite people to their accounts, as well as assign who has editing privileges and who has read-only access.

Who has access to my video and audio files that have been indexed and/or stored by Azure Video Indexer and the metadata and insights that were extracted?

Your video or audio content that have public as its privacy setting can be accessed by anyone who has the link to your video or audio content and its insights. Your video or audio content that have private as its privacy setting can only be accessed by users that were invited to the account of the video or audio content. The privacy setting of your content also applies to the metadata and insights that Azure Video Indexer extracts. You assign the privacy setting when you upload your video or audio file. You can also change the privacy setting after indexing.

What access does Microsoft have to my video or audio files that have been indexed and/or stored by Azure Video Indexer and the metadata and insights that were extracted?

Per the Azure Online Services Terms (OST), you completely own your content, and Microsoft will only access your content and the metadata and insights that Azure Video Indexer extracts from your content according to the OST and the Microsoft Privacy Statement.

Are the custom models that I build in my Azure Video Indexer account available to other accounts?

No, the custom models that you create in your account are not available to any other account. Azure Video Indexer currently allows you to build custom brands, language, and person models in your account. These models are only available in the account in which you created the models.

Is the content indexed by Video Indexer kept within the Azure region where I am using Video Indexer?

Yes, the content and its insights are kept within the Azure region (except for Singapore and Brazil South regions) unless you have a manual configuration in your Azure subscription that uses multiple Azure regions.

Customer data in a region is replicated for BCDR reasons to the paired region.

What is the privacy policy for Azure Video Indexer?

Azure Video Indexer is covered by the Microsoft Privacy Statement. The privacy statement explains the personal data Microsoft processes, how Microsoft processes it, and for what purposes Microsoft processes it. To learn more about privacy, visit the Microsoft Trust Center.

What certifications does Azure Video Indexer have?

Azure Video Indexer currently has the SOC certification. To review Azure Video Indexer's certification, please refer to the Microsoft Trust Center.

What is the difference between private and public videos?

When videos are uploaded to Azure Video Indexer, you can choose from two privacy settings: private and public. Public videos are accessible for anyone, including anonymous and unidentified users. Private ones are restricted solely to the account members.

I tried to upload a video as public and it was flagged for inappropriate or offensive content, what does that mean?

When uploading a video to Azure Video Indexer, an automatic content analysis is done by algorithms and models in order to make sure no inappropriate content will be presented publicly. If a video is found to be suspicious as containing explicit content, it will not be possible to set it as public. However, the account members can still access it as a private video (view it, download the insights and extracted artifacts, and perform other operations available to account members).

Warning

We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

In order to set the video for public access, you can either:

  • Build your own interface layer (such as app or website) and use it to interact with the Azure Video Indexer service. This way the video remains private in our website and your users can interact with it through your interface. For example, you can still get the insights or allow viewing of the video in your own interface.

  • Request a human review of the content, which would result in removing of the restriction assuming the content is not explicit.

    This option can be explored if the Azure Video Indexer website is used directly by your users as the interface layer, and for public (unauthenticated) viewing.

API Questions

What APIs does Azure Video Indexer offer?

Azure Video Indexer's APIs allows for indexing, extracting metadata, asset management, translation, embedding, customization of models and more. To find more detailed information on using the Azure Video Indexer API, refer to the Azure Video Indexer developer portal.

What client SDKs does Azure Video Indexer offer?

There are currently no client SDKs offered. The Azure Video Indexer team is working on the SDKs and plans to deliver them soon.

How do I get started with Azure Video Indexer's API?

What is the difference between the Azure Video Indexer API and the Azure Media Service v3 API?

Currently there are some overlaps in features offered by the Azure Video Indexer API and the Azure Media Service v3 API. You can find more information on how to compare both services here.

What is an API access token and why do I need it?

The Azure Video Indexer API contains an Authorization API and an Operations API. The Authorizations API contains calls that give you access token. Each call to the Operations API should be associated with an access token, matching the authorization scope of the call.

Access tokens are needed to use the Azure Video Indexer APIs for security purposes. This ensures that any calls are coming from you or those who have access permissions to your account. 

What is the difference between Account access token, User access token, and Video access token?

  • Account level – account level access tokens let you perform operations on the account level or the video level. For example, upload a video, list all videos, get video insights.
  • User level - user level access tokens let you perform operations on the user level. For example, get associated accounts.
  • Video level – video level access tokens let you perform operations on a specific video. For example, get video insights, download captions, get widgets, etc.

How often do I need to get a new access token? When do access tokens expire?

Access tokens expire every hour, so you need to generate a new access token every hour.

What are the login options to Azure Video Indexer Developer portal?

See a release note regarding login information.

Once you register your email account using an identity provider, you cannot use this email account with another identity provider.

Billing questions

How much does Azure Video Indexer cost?

Azure Video Indexer uses a simple pay-as-you-go pricing model based on the duration of the content input that you index. Additional charges may apply for encoding, streaming, storage, network usage, and media reserved units. For more information, see the pricing page.

When am I billed for using Azure Video Indexer?

When sending a video to be indexed, the user will define the indexing to be video analysis, audio analysis or both. This will determine which SKUs will be charged. If there is a critical level error during processing, an error code will be returned as a response. In such a case, no billing occurs. A critical error can be caused by a bug in our code or a critical failure in an internal dependency the service has. Errors such as wrong identification or insight extraction are not considered as critical and a response is returned. In any case where a valid (non-error code) response is returned, billing occurs.

Does Azure Video Indexer offer a trial account?

Yes, Azure Video Indexer offers a trial account that gives full service and API functionality. There is a quota of 600 minutes worth of videos for web-based interface users and 2,400 minutes for API users.