Azure AI Video Indexer frequently asked questions

This article answers frequently asked questions about Azure AI Video Indexer.

General questions

What is Azure AI Video Indexer?

Azure AI Video Indexer is a cloud application, part of Azure AI services, enables you to extract the insights from your videos using Azure AI Video Indexer video and audio models. For more information, see Overview.

What can I do with Azure AI Video Indexer?

Some of the operations that Azure AI Video Indexer can perform on media files include:

  • Identifying and extracting speech and identify speakers.
  • Identifying and extracting on-screen text in a video.
  • Detecting objects in a video file.
  • Identify brands (for example: Microsoft) from audio tracks and on-screen text in a video.
  • Detecting and recognizing faces from a database of celebrities and a user-defined database of faces.
  • Extracting topics discussed but not necessarily mentioned in audio and video content.
  • Creating closed captions or subtitles from the audio track.

For more information and more Azure AI Video Indexer features, see Overview.

How do I get started with Azure AI Video Indexer?

Azure AI Video Indexer includes a trial offering that provides you with 2,400 free indexing minutes when using the Azure AI Video Indexer website or the Azure AI Video Indexer API (see developer portal).

To index videos and audio flies at scale, you can connect Azure AI Video Indexer to a paid Microsoft Azure subscription. You can find more information on pricing on the pricing page.

You can find more information on getting started in Get started.

Do I need coding skills to use Azure AI Video Indexer?

You can use the Azure AI Video Indexer web-based interface to evaluate, configure, and manage your account with no coding required. When you are ready to develop more complex applications, you can use the Azure AI Video Indexer API to integrate Azure AI Video Indexer into your own applications, web sites, or custom workflows using serverless technologies like Azure Logic Apps or Azure Functions.

What are the Limited Access features of Azure AI Video Indexer and how to register?

Face Identify and Celebrity Recognition features of Azure AI Video Indexer are Limited Access features that require registration.

Find the registration form and more information about Limited Access in this article.

Also, see the following: the announcement blog post and investment and safeguard for facial recognition blog post.

Do I need machine learning skills to use Azure AI Video Indexer?

No, Azure AI Video Indexer provides the integration of multiple machine learning models into one pipeline. Indexing a video or audio file via Azure AI Video Indexer retrieves a full set of insights extracted on one shared timeline without any machine learning skills or knowledge on algorithms needed on the customer's part.

What media formats does Azure AI Video Indexer support?

Azure AI Video Indexer supports most common media formats. See Azure AI Video Indexer support matrix and service limits.

How do I upload a media file into Azure AI Video Indexer and what are the limitations?

In the Azure AI Video Indexer web-based portal, you can upload a media file using the file upload dialog or by pointing to a URL that directly hosts the source file. Any URL that hosts the media content using an iFrame or embed code will not work (see example).

For more information, please see this how-to guide.

Can I stream a video using an external HTML5 video player

Yes. To access the raw video file from video indexer you can use the Get Video Source File Download Url method to get the MP4 file URL and put it in the video tag.

How many files can I stitch and render in a project?

In Azure AI Video Indexer you can create a project and add multiple files to stitch and render as a new file. The number of source files is set to 10 in the website and 100 in the API.

How long does it take Azure AI Video Indexer to extract insights from media?

The amount of time it takes to index a video or audio file, both using the Azure AI Video Indexer API and the Azure AI Video Indexer web-based interface, depends on multiple parameters such as the file length and quality, the number of insights found in the file, among other things. We recommend that you run a few test files with your own content and take an average to get a better idea.

Can I create customized workflows to automate processes with Azure AI Video Indexer?

Yes, you can integrate Azure AI Video Indexer into serverless technologies like Logic Apps, Flow, and Azure Functions. You can find more details on the Logic App and Flow connectors for Azure AI Video Indexer here. You can see some automation projects done by partners in the Azure AI Video Indexer Samples repo.

In which Azure regions is Azure AI Video Indexer available?

You can see which Azure regions Azure AI Video Indexer is available on the regions page.

Can I customize Azure AI Video Indexer models for my specific use case?

Yes. In Azure AI Video Indexer you can customize some of the available models to better fit your needs.

For example, our Person model supports out-of-the-box 1,000,000 faces of celebrity recognition, but you can also train it to recognize other faces which are not in that database.

For details, see articles about customizing Person, Brands, and Language models.

Can I edit the videos in my library?

Yes. Press the edit video button from the library display or the open in editor button from the player display to get to the Projects tab. You can create a new project and add more videos from your library to edit them together, once you are done you can render your video and download.

If you want to get insights on your new video, index it with Azure AI Video Indexer and it will appear in your library with its insights.

Can I index multiple audio streams or channels?

If there are multiple audio streams, Azure AI Video Indexer takes the first one it encounters and will process only this stream. In any audio stream Azure AI Video Indexer processes, it takes the different channels (if present) and processes them together as mono. For streams/channels manipulation you can use ffmpeg commands on the file before indexing it.

Can’t find my new ARM account in the account list on the Azure AI Video Indexer website.

For detailed description on how to switch to the correct domain, see Switch between tenants.

Azure AI Video Indexer enabled by Arc questions

What is Azure AI Video Indexer Arc Enabled Extension?

An application running in Arc-enabled Kubernetes containers that can be deployed outside of Azure to extract and consume audio insights from your media files. Future versions will provide video insights as well.

What is Azure Arc?

Azure Arc is a bridge that extends the Azure platform to help you build applications and services with the flexibility to run across datacenters, at the edge, and in multi-cloud environments. Develop cloud-native applications with consistent development, operations, and security models.

What is needed to participate in the preview?

To participate in the preview, you need an active Azure subscription, a connection to Azure, an environment that is capable of running the extension (see [requirements in the quickstart, and permission to join the preview. You can submit a request to join the preview here.

How can I get started with Azure AI Video Indexer enabled by Arc?

You can either follow the quickstart and experience Azure Arc-enabled Kubernetes automated scenarios with Azure Arc Jumpstart.

Why is the download so large?

During the installation of the extension, machine learning containers are downloaded to the edge compute. Some of these containers can be several gigabytes in size, and the download time will depend on network conditions. In some cases, the download may take longer than 15 minutes.

Do I need to configure storage to my Kubernetes cluster?

Video Indexer enabled by Arc requires a storage class configured with readwritemany. If a storage class is already set as the cluster’s default during installation, then it will be used. Otherwise, if you need to configure different storage, you can pass it as a property during the extension installation.

Do I need to configure a frontend endpoint for Video Indexer extension?

Yes. The Video Indexer extension’s API is available via the frontend extension. You can restrict access to either an IP address or a FQDN. Also, as all communication is done over secure traffic protocols, using a valid CA certificate, and setting a security layer such as NSG or firewall to limit the access to this endpoint is recommended.

How can I troubleshoot my Azure AI Video Indexer enabled by Arc installation?

Consult the troubleshooting guide.

Privacy Questions

How is network traffic encrypted by Azure AI Video Indexer?

Communication between a client application and an Azure AI Video Indexer account is encrypted using Transport Layer Security (TLS). TLS is a standard cryptographic protocol that ensures privacy and data integrity between clients and services over the Internet. For more information about TLS, see Transport Layer Security. Video Indexer enforces a minimum of TLS v1.2. Customers can't control the cipher suits, we will pick the first match between the client and server going from strong to weaker.

Are video and audio files indexed by Azure AI Video Indexer stored?

Yes, unless you delete the file from Azure AI Video Indexer, either using the Azure AI Video Indexer website or API, your video and audio files are stored. For the trial, the video and audio files that you index are stored in the Azure region East US. Otherwise, your video and audio files are stored in the storage account of your Azure subscription.

Where does Azure AI Video Indexer store customer data?

By default, your data is replicated to the paired region into which you deployed the Azure AI Video Indexer instance. For region with data residency requirements, your data is always kept within the same region. For more related information, check out other Privacy Questions in this section.

Can I delete my files that are stored in the Azure AI Video Indexer website?

Yes, you can always delete your video and audio files as well as any metadata and insights extracted from them by Azure AI Video Indexer. Once you delete a file from Azure AI Video Indexer, the file and its metadata and insights are permanently removed from Azure AI Video Indexer. However, if you have implemented your own backup solution in Azure storage, the file remains in your Azure storage.

Can I control user access to my Azure AI Video Indexer account?

Yes, only account admins can invite and uninvite people to their accounts, as well as assign who has editing privileges and who has read-only access.

Who has access to my video and audio files that have been indexed and/or stored by Azure AI Video Indexer and the metadata and insights that were extracted?

Your video or audio content that have public as its privacy setting can be accessed by anyone who has the link to your video or audio content and its insights. Your video or audio content that have private as its privacy setting can only be accessed by users that were invited to the account of the video or audio content. The privacy setting of your content also applies to the metadata and insights that Azure AI Video Indexer extracts. You assign the privacy setting when you upload your video or audio file. You can also change the privacy setting after indexing.

What access does Microsoft have to my video or audio files that have been indexed and/or stored by Azure AI Video Indexer and the metadata and insights that were extracted?

Per the Azure Online Services Terms (OST), you completely own your content, and Microsoft will only access your content and the metadata and insights that Azure AI Video Indexer extracts from your content according to the OST and the Microsoft Privacy Statement.

Are the custom models that I build in my Azure AI Video Indexer account available to other accounts?

No, the custom models that you create in your account are not available to any other account. Azure AI Video Indexer currently allows you to build custom brands, language, and person models in your account. These models are only available in the account in which you created the models.

Is the content indexed by Video Indexer kept within the Azure region where I am using Video Indexer?

Yes, the content and its insights are kept within the Azure region (except for Singapore and Brazil South regions) unless you have a manual configuration in your Azure subscription that uses multiple Azure regions.

Customer data in a region is replicated for BCDR reasons to the paired region.

What is the privacy policy for Azure AI Video Indexer?

Azure AI Video Indexer is covered by the Microsoft Privacy Statement. The privacy statement explains the personal data Microsoft processes, how Microsoft processes it, and for what purposes Microsoft processes it. To learn more about privacy, visit the Microsoft Trust Center.

What certifications does Azure AI Video Indexer have?

Azure AI Video Indexer currently has the SOC certification. To review Azure AI Video Indexer's certification, please refer to the Microsoft Trust Center.

What is the difference between private and public videos?

When videos are uploaded to Azure AI Video Indexer, you can choose from two privacy settings: private and public. Public videos are accessible for anyone, including anonymous and unidentified users. Private ones are restricted solely to the account members.

I tried to upload a video as public and it was flagged for inappropriate or offensive content, what does that mean?

When uploading a video to Azure AI Video Indexer, an automatic content analysis is done by algorithms and models in order to make sure no inappropriate content will be presented publicly. If a video is found to be suspicious as containing explicit content, it will not be possible to set it as public. However, the account members can still access it as a private video (view it, download the insights and extracted artifacts, and perform other operations available to account members).

Warning

We do not recommend that you use data directly from the artifacts folder for production purposes. Artifacts are intermediate outputs of the indexing process. They are essentially raw outputs of the various AI engines that analyze the videos; the artifacts schema may change over time. It is recommended that you use the Get Video Index API, as described in Get insights and artifacts produced by the API.

In order to set the video for public access, you can either:

  • Build your own interface layer (such as app or website) and use it to interact with the Azure AI Video Indexer service. This way the video remains private in our website and your users can interact with it through your interface. For example, you can still get the insights or allow viewing of the video in your own interface.

  • Request a human review of the content, which would result in removing of the restriction assuming the content is not explicit.

    This option can be explored if the Azure AI Video Indexer website is used directly by your users as the interface layer, and for public (unauthenticated) viewing.

API Questions

What APIs does Azure AI Video Indexer offer?

Azure AI Video Indexer's APIs allows for indexing, extracting metadata, asset management, translation, embedding, customization of models and more. To find more detailed information on using the Azure AI Video Indexer API, refer to the Azure AI Video Indexer developer portal.

What client SDKs does Azure AI Video Indexer offer?

There are currently no client SDKs offered. The Azure AI Video Indexer team is working on the SDKs and plans to deliver them soon.

How do I get started with Azure AI Video Indexer's API?

What is an API access token and why do I need it?

The Azure AI Video Indexer API contains an Authorization API and an Operations API. The Authorizations API contains calls that give you access token. Each call to the Operations API should be associated with an access token, matching the authorization scope of the call.

Access tokens are needed to use the Azure AI Video Indexer APIs for security purposes. This ensures that any calls are coming from you or those who have access permissions to your account.

What is the difference between Account access token, User access token, and Video access token?

  • Account level – account level access tokens let you perform operations on the account level or the video level. For example, upload a video, list all videos, get video insights.
  • User level - user level access tokens let you perform operations on the user level. For example, get associated accounts.
  • Video level – video level access tokens let you perform operations on a specific video. For example, get video insights, download captions, get widgets, etc.

How often do I need to get a new access token? When do access tokens expire?

Access tokens expire every hour, so you need to generate a new access token every hour.

What are the login options to Azure AI Video Indexer Developer portal?

See a release note regarding login information.

Once you register your email account using an identity provider, you cannot use this email account with another identity provider.

Where are the JSON assets (like insights, transcript, etc.), associated with indexed videos, stored?

They are stored by the Azure AI Video Indexer service and can be fetched with the Get Video Index and Get Video Artifact Download Url APIs.

Once a video has been indexed, are there any additional charges for calling the API?

There are no additional charges from Azure AI Video Indexer for calling the API.

Billing questions

How much does Azure AI Video Indexer cost?

Azure AI Video Indexer uses a simple pay-as-you-go pricing model based on the duration of the content input that you index. Additional charges may apply for encoding, streaming, storage, network usage, and media reserved units. For more information, see the pricing page.

When am I billed for using Azure AI Video Indexer?

When sending a video to be indexed, the user will define the indexing to be video analysis, audio analysis or both. This will determine which SKUs will be charged. If there is a critical level error during processing, an error code will be returned as a response. In such a case, no billing occurs. A critical error can be caused by a bug in our code or a critical failure in an internal dependency the service has. Errors such as wrong identification or insight extraction are not considered as critical and a response is returned. In any case where a valid (non-error code) response is returned, billing occurs.

Does Azure AI Video Indexer offer a trial account?

Yes, Azure AI Video Indexer offers a trial account that that provides you with 2,400 free indexing minutes when using the Azure AI Video Indexer website or the Azure AI Video Indexer API (see developer portal).

How to opt out results for specific AI models

How to opt out a specific result from a public AI model?

Send a detailed explanation of your request to "Video Indexer Customer Support" visupport@microsoft.com. Including: AI model you are referring to, the result you'd like to opt out, the reason we should opt out the results and an explanation of why you are asking this.

How long does it take?

Processing the request might take few weeks.