CLI example: Run a job and tasks with Azure Batch

This script creates a Batch job and adds a series of tasks to the job. It also demonstrates how to monitor a job and its tasks.

If you don't have an Azure subscription, create an Azure free account before you begin.

Prerequisites

• Use the Bash environment in Azure Cloud Shell. For more information, see Quickstart for Bash in Azure Cloud Shell.

  

• If you prefer to run CLI reference commands locally, install the Azure CLI. If you're running on Windows or macOS, consider running Azure CLI in a Docker container. For more information, see How to run the Azure CLI in a Docker container.

  • If you're using a local installation, sign in to the Azure CLI by using the az login command. To finish the authentication process, follow the steps displayed in your terminal. For other sign-in options, see Sign in with the Azure CLI.

  • When you're prompted, install the Azure CLI extension on first use. For more information about extensions, see Use extensions with the Azure CLI.

  • Run az version to find the version and dependent libraries that are installed. To upgrade to the latest version, run az upgrade.

Sample script

Launch Azure Cloud Shell

The Azure Cloud Shell is a free interactive shell that you can use to run the steps in this article. It has common Azure tools preinstalled and configured to use with your account.

To open the Cloud Shell, just select Try it from the upper right corner of a code block. You can also launch Cloud Shell in a separate browser tab by going to https://shell.azure.com.

When Cloud Shell opens, verify that Bash is selected for your environment. Subsequent sessions will use Azure CLI in a Bash environment, Select Copy to copy the blocks of code, paste it into the Cloud Shell, and press Enter to run it.

Sign in to Azure

Cloud Shell is automatically authenticated under the initial account signed-in with. Use the following script to sign in using a different subscription, replacing <Subscription ID> with your Azure Subscription ID. If you don't have an Azure subscription, create an Azure free account before you begin.

subscription="<subscriptionId>" # add subscription here

az account set -s $subscription # ...or use 'az login'

For more information, see set active subscription or log in interactively

Create a Batch account in Batch service mode

# Run a job and tasks with Azure Batch

# Variable block
let "randomIdentifier=$RANDOM*$RANDOM"
location="East US"
[[ "$RESOURCE_GROUP" == '' ]] && resourceGroup="msdocs-batch-rg-$randomIdentifier" || resourceGroup="${RESOURCE_GROUP}"
tag="run-job"
storageAccount="msdocsstorage$randomIdentifier"
batchAccount="msdocsbatch$randomIdentifier"

# Create a resource group.
echo "Creating $resourceGroup in "$location"..."
az group create --name $resourceGroup --location "$location" --tag $tag

# Create a general-purpose storage account in your resource group.
echo "Creating $storageAccount"
az storage account create --resource-group $resourceGroup --name $storageAccount --location "$location" --sku Standard_LRS

# Create a Batch account.
echo "Creating $batchAccount"
az batch account create --name $batchAccount --storage-account $storageAccount --resource-group $resourceGroup --location "$location"

# Authenticate against the account directly for further CLI interaction.
az batch account login --name $batchAccount --resource-group $resourceGroup --shared-key-auth

# Create a new Linux pool with a virtual machine configuration. 
az batch pool create --id mypool --vm-size Standard_A1 --target-dedicated 2 --image canonical:ubuntuserver:18_04-lts-gen2 --node-agent-sku-id "batch.node.ubuntu 18.04"

# Create a new job to encapsulate the tasks that are added.
az batch job create --id myjob --pool-id mypool

# Add tasks to the job. Here the task is a basic shell command.
az batch task create --job-id myjob --task-id task1 --command-line "/bin/bash -c 'printenv AZ_BATCH_TASK_WORKING_DIR'"

To add many tasks at once

To add many tasks at once, specify the tasks in a JSON file, and pass it to the command. For format, see https://github.com/Azure/azure-docs-cli-python-samples/blob/master/batch/run-job/tasks.json. Provide the absolute path to the JSON file. For an example JSON file, see https://github.com/Azure-Samples/azure-cli-samples/blob/master/batch/run-job/tasks.json.

az batch task create \
    --job-id myjob \
    --json-file tasks.json

To update the job

Update the job so that it is automatically marked as completed once all the tasks are finished.

az batch job set \
--job-id myjob \
--on-all-tasks-complete terminatejob

To monitor the status of the job

az batch job show --job-id myjob

To monitor the status of a task

az batch task show \
    --job-id myjob \
    --task-id task1

Clean up resources

Use the following command to remove the resource group and all resources associated with it using the az group delete command - unless you have an ongoing need for these resources. Some of these resources may take a while to create, as well as to delete.

az group delete --name $resourceGroup

Sample reference

This script uses the following commands. Each command in the table links to command-specific documentation.

Command	Notes
az group create	Creates a resource group in which all resources are stored.
az batch account create	Creates the Batch account.
az batch account login	Authenticates against the specified Batch account for further CLI interaction.
az batch pool create	Creates a pool of compute nodes.
az batch job create	Creates a Batch job.
az batch task create	Adds a task to the specified Batch job.
az batch job set	Updates properties of a Batch job.
az batch job show	Retrieves details of a specified Batch job.
az batch task show	Retrieves the details of a task from the specified Batch job.
az group delete	Deletes a resource group including all nested resources.

Next steps

For more information on the Azure CLI, see Azure CLI documentation.