Deploy and run containers on Azure Container Instance

With the following steps, scale Azure AI services applications in the cloud easily with Azure Container Instances. Containerization helps you focus on building your applications instead of managing the infrastructure. For more information on using containers, see features and benefits.

Prerequisites

The recipe works with any Azure AI services container. The Azure AI services resource must be created before using the recipe. Each Azure AI service that supports containers has a "How to install" article for installing and configuring the service for a container. Some services require a file or set of files as input for the container, it is important that you understand and have used the container successfully before using this solution.

• An Azure resource for the Azure AI service you're using.

• Azure AI service resource endpoint URL - review your specific service's "How to install" for the container, to find where the endpoint URL is from within the Azure portal, and what a correct example of the URL looks like. The exact format can change from service to service.

• Azure AI service resource key - the keys are on the Keys page for the Azure resource. You only need one of the two keys. The key is a string of 32 alpha-numeric characters.

• A single Azure AI services container on your local host (your computer). Make sure you can:

  • Pull down the image with a docker pull command.
  • Run the local container successfully with all required configuration settings with a docker run command.
  • Call the container's endpoint, getting a response of HTTP 2xx and a JSON response back.

All variables in angle brackets, <>, need to be replaced with your own values. This replacement includes the angle brackets.

Important

The LUIS container requires a .gz model file that is pulled in at runtime. The container must be able to access this model file via a volume mount from the container instance. To upload a model file, follow these steps:

1. Create an Azure file share. Take note of the Azure Storage account name, key, and file share name as you'll need them later.
2. export your LUIS model (packaged app) from the LUIS portal.
3. In the Azure portal, navigate to the Overview page of your storage account resource, and select File shares.
4. Select the file share name that you recently created, then select Upload. Then upload your packaged app.

Azure portal

Create an Azure Container Instance resource using the Azure portal

1. Go to the Create page for Container Instances.

2. On the Basics tab, enter the following details:

   Setting	Value
   Subscription	Select your subscription.
   Resource group	Select the available resource group or create a new one such as cognitive-services.
   Container name	Enter a name such as cognitive-container-instance. The name must be in lower caps.
   Location	Select a region for deployment.
   Image type	If your container image is stored in a container registry that doesn’t require credentials, choose Public. If accessing your container image requires credentials, choose Private. Refer to container repositories and images for details on whether or not the container image is Public or Private ("Public Preview").
   Image name	Enter the Azure AI services container location. The location is what's used as an argument to the docker pull command. Refer to the container repositories and images for the available image names and their corresponding repository. The image name must be fully qualified specifying three parts. First, the container registry, then the repository, finally the image name: <container-registry>/<repository>/<image-name>. Here is an example, mcr.microsoft.com/azure-cognitive-services/keyphrase would represent the Key Phrase Extraction image in the Microsoft Container Registry under the Azure AI services repository. Another example is, containerpreview.azurecr.io/microsoft/cognitive-services-speech-to-text which would represent the Speech to text image in the Microsoft repository of the Container Preview container registry.
   OS type	Linux
   Size	Change size to the suggested recommendations for your specific Azure AI container: 2 CPU cores 4 GB

3. On the Networking tab, enter the following details:

   Setting	Value
   Ports	Set the TCP port to 5000. Exposes the container on port 5000.

4. On the Advanced tab, enter the required Environment Variables for the container billing settings of the Azure Container Instance resource:

   Key	Value
   ApiKey	Copied from the Keys and endpoint page of the resource. It is a 32 alphanumeric-character string with no spaces or dashes, xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx.
   Billing	Your endpoint URL copied from the Keys and endpoint page of the resource.
   Eula	accept

5. Select Review and Create

6. After validation passes, click Create to finish the creation process

7. When the resource is successfully deployed, it's ready

CLI

Create an Azure Container Instance resource from the Azure CLI

The YAML below defines the Azure Container Instance resource. Copy and paste the contents into a new file, named my-aci.yaml and replace the commented values with your own. Refer to the template format for valid YAML. Refer to the container repositories and images for the available image names and their corresponding repository. For more information of the YAML reference for Container instances, see YAML reference: Azure Container Instances.

apiVersion: 2018-10-01
location: # < Valid location >
name: # < Container Group name >
properties:
  imageRegistryCredentials: # This is only required if you are pulling a non-public image that requires authentication to access. For example Text Analytics for health.
  - server: containerpreview.azurecr.io
    username: # < The username for the preview container registry >
    password: # < The password for the preview container registry >
  containers:
  - name: # < Container name >
    properties:
      image: # < Repository/Image name >
      environmentVariables: # These env vars are required
        - name: eula
          value: accept
        - name: billing
          value: # < Service specific Endpoint URL >
        - name: apikey
          value: # < Service specific API key >
      resources:
        requests:
          cpu: 4 # Always refer to recommended minimal resources
          memoryInGb: 8 # Always refer to recommended minimal resources
      ports:
        - port: 5000
  osType: Linux
  volumes: # This node, is only required for container instances that pull their model in at runtime, such as LUIS.
  - name: aci-file-share
    azureFile:
      shareName: # < File share name >
      storageAccountName: # < Storage account name>
      storageAccountKey: # < Storage account key >
  restartPolicy: OnFailure
  ipAddress:
    type: Public
    ports:
    - protocol: tcp
      port: 5000
tags: null
type: Microsoft.ContainerInstance/containerGroups

Note

Not all locations have the same CPU and Memory availability. Refer to the location and resources table for the listing of available resources for containers per location and OS.

We'll rely on the YAML file we created for the az container create command. From the Azure CLI, execute the az container create command replacing the <resource-group> with your own. Additionally, for securing values within a YAML deployment refer to secure values.

az container create -g <resource-group> -f my-aci.yaml

The output of the command is Running... if valid, after sometime the output changes to a JSON string representing the newly created ACI resource. The container image is more than likely not be available for a while, but the resource is now deployed.

Tip

Pay close attention to the locations of Azure AI services in public preview, as the YAML will needed to be adjusted accordingly to match the location.

Use the Container Instance

Azure portal

1. Select the Overview and copy the IP address. It will be a numeric IP address such as 55.55.55.55.

2. Open a new browser tab and use the IP address, for example, http://<IP-address>:5000 (http://55.55.55.55:5000). You will see the container's home page, letting you know the container is running.

   

3. Select Service API Description to view the swagger page for the container.

4. Select any of the POST APIs and select Try it out. The parameters are displayed including the input. Fill in the parameters.

5. Select Execute to send the request to your Container Instance.

   You have successfully created and used Azure AI containers in Azure Container Instance.

CLI

Validate that a container is running

There are several ways to validate that the container is running. Locate the External IP address and exposed port of the container in question, and open your favorite web browser. Use the various request URLs that follow to validate the container is running. The example request URLs listed here are http://localhost:5000, but your specific container might vary. Make sure to rely on your container's External IP address and exposed port.

Request URL	Purpose
http://localhost:5000/	The container provides a home page.
http://localhost:5000/ready	Requested with GET, this URL provides a verification that the container is ready to accept a query against the model. This request can be used for Kubernetes liveness and readiness probes.
http://localhost:5000/status	Also requested with GET, this URL verifies if the api-key used to start the container is valid without causing an endpoint query. This request can be used for Kubernetes liveness and readiness probes.
http://localhost:5000/swagger	The container provides a full set of documentation for the endpoints and a Try it out feature. With this feature, you can enter your settings into a web-based HTML form and make the query without having to write any code. After the query returns, an example CURL command is provided to demonstrate the HTTP headers and body format that's required.

Note

If you're running the Text Analytics for health container, use the following URL to submit queries: http://localhost:5000/text/analytics/v3.2-preview.1/entities/health