Batch Service REST API Versioning

  • Article

Operations provided by the Batch service REST API may have multiple versions for backwards compatibility as the API evolves over time. You must specify which version of an operation you wish to use when it is called by providing the version with your REST call. If your application calls an older version of an operation, you can choose to continue calling the older version, or modify your code to call a newer version. If the version is not specified or an incorrect version is specified, then an error is returned.

To specify which version of an operation to use, set the api-version query parameter. The version is of the format Group.Major.Minor where Group is in the format ‘YYYY-MM-DD’ and Major is an integer and Minor is an integer.

Latest version: 2023-05-01.17.0

  • [Breaking] Removed Batch Job/Pool Lifetime Statistics API: lifetimejobstats and lifetimepoolstats.
  • Added boolean property enableAcceleratedNetworking to NetworkConfiguration.
    • This property determines whether this pool should enable accelerated networking, with default value as False.
    • Whether this feature can be enabled is also related to whether an operating system/VM instance is supported, which should align with AcceleratedNetworking Policy.
  • Added boolean property enableAutomaticUpgrade to VMExtension.
    • This property determines whether the extension should be automatically upgraded by the platform if there is a newer version of the extension available.
  • The property type of ContainerConfiguration, which now supports two values: DockerCompatible and CriCompatible.

Previous Versions

Previous versions include:

Version: 2022-10-01.16.0

  • Added new NodeCommunicationMode property which determines how a pool communicates with the Batch service
  • Added new currentNodeCommunicationMode property of the NodeCommunicationMode type to the CloudPool model to refer to the current state of the pool communication mode.
  • Added new targetNodeCommunicationMode property of the NodeCommunicationMode type to the following models to refer to the desired node communication mode for the pool:

Version: 2022-01-01.15.0

Version: 2021-06-01.14.0

  • Add ability to assign user-assigned managed identities to CloudPool. These identities will be made available on each node in the pool, and can be used to access various resources.
  • Added identityReference property to the following models to support accessing resources via managed identity:
  • Added new extensions property to VirtualMachineConfiguration on CloudPool to specify virtual machine extensions for nodes
  • Added the ability to specify availability zones using a new property nodePlacementConfiguration on VirtualMachineConfiguration
  • Added new osDisk property to VirtualMachineConfiguration, which contains settings for the operating system disk of the Virtual Machine.
    • The placement property on DiffDiskSettings specifies the ephemeral disk placement for operating system disks for all VMs in the pool. Setting it to "CacheDisk" will store the ephemeral OS disk on the VM cache.
  • Added maxParallelTasks property on CloudJob to control the maximum number of tasks that can run at the same time per job (defaults to -1 meaning unlimited). This property is currently in private preview.
  • Added virtualMachineInfo property on ComputeNode which contains information about the current state of the virtual machine, including the exact version of the marketplace image the VM is using.
  • Added recurrenceInterval property to Schedule to control the interval between the start times of two successive jobs under a job schedule.

Version: 2020-09-01.12.0

  • [Breaking] Replaced property maxTasksPerNode with taskSlotsPerNode on the pool. Using this property tasks in a job can consume a dynamic number of slots allowing for more fine-grained control over resource consumption.
  • [Breaking] Changed the response type of GetTaskCounts to return TaskCountsResult, which is a complex object containing the previous TaskCounts object and a new TaskSlotCounts object providing similar information in the context of slots being used.
  • Added property requiredSlots to the task allowing user to specify how many slots on a node it should take up.

Version: 2020-03-01.11.0

Version: 2019-08-01.10.0

  • Added ability to specify a collection of public IPs on NetworkConfiguration via the new publicIPs property on the pool. This guarantees nodes in the pool will have an IP from the list user provided IPs.
  • Added ability to mount remote file-systems on each node of a pool via the MountConfiguration property.
  • Shared Image Gallery images can now be specified on the virtualMachineImageId property of ImageReference by referencing the image via its Azure Resource Manager ID.
  • [Breaking] When not specified, the default value for waitForSuccess on StartTask is true (previously, it was false).
  • [Breaking] When not specified, the default value for scope on AutoUserSpecification is now always Pool (previously, it was Task on Windows nodes and Pool on Linux nodes).
  • Improved various confusing or incomplete documents.

Version: 2019-06-01.9.0

  • [Breaking] Replaced ListNodeAgentSKUs with ListSupportedImages. ListSupportedImages contains all of the same information originally available in ListNodeAgentSKUs but in a clearer format. New, non-verified images are also now returned. Additional information about Capabilities and BatchSupportEndOfLife is accessible via ImageInformation.

  • Now support network security rules blocking network access to a CloudPool based on the source port of the traffic. This is done via the SourcePortRanges property on NetworkSecurityGroupRule.

  • When running a container, Batch now supports executing the task in the container working directory or in the Batch task working directory. This is controlled by the WorkingDirectory property on TaskContainerSettings.

  • Improved various confusing or incomplete documents.

Version: 2018-12-01.8.0

  • [Breaking] Removed support for upgradeos on cloudServiceConfiguration pools.

    • Removed upgradeos.
    • Renamed targetOSVersion to osVersion and removed currentOSVersion on CloudPool.
    • Removed upgrading from PoolState.

  • [Breaking] Removed dataEgressGiB and dataIngressGiB from poolusagemetrics. These properties are no longer supported.

  • [Breaking] ResourceFile improvements

    • Added the ability specify an entire Azure Storage container in ResourceFile. There are now three supported modes for ResourceFile:
      • httpUrl creates a ResourceFile pointing to a single HTTP URL.
      • storageContainerUrl creates a ResourceFile pointing to an Azure Blob Storage container.
      • autoStorageContainerName creates a ResourceFile pointing to an Azure Blob Storage container in the Batch registered autostorage account.
    • URLs provided to ResourceFile via the httpUrl method can now be any HTTP URL. Previously, these had to be an Azure Blob Storage URL.
    • blobPrefix can be used to filter downloads from a storage container to only those matching the prefix.

  • [Breaking] Removed osDisk property from VirtualMachineConfiguration. This property is no longer supported.

  • Pools that set the dynamicVNetAssignmentScope on NetworkConfiguration to be job can now dynamically assign a Virtual Network to each node the job's tasks run on. The specific Virtual Network to join the nodes to is specified in the new JobNetworkConfiguration property on CloudJob and JobSpecification.

    Note

    This feature is in public preview. It is disabled for all Batch accounts except for those which have contacted us and requested to be in the pilot.

  • The maximum lifetime of a task is now 180 days (previously it was 7).

  • Added support on Windows pools for creating users with a specific login mode (either batch or interactive) via LoginMode.

  • The default task retention time for all tasks is now 7 days, previously it was infinite.

Version 2018-08-01.7.0

New features in version 2018-08-01.7.0 include:

  • View the version of the Azure Batch Node Agent, via the new NodeAgentInfo property on ComputeNode.
  • Added the ability to specify a Filter on the Result of a task. See here for more details.
    • This enables the often requested scenario of performing a server-side query to find all tasks that failed.
  • [Breaking] Removed the ValidationStatus property from TaskCounts.
  • [Breaking] The default caching type for DataDisk and OSDisk is now ReadWrite instead of None.
  • [Breaking] Renamed the only value of ContainerType from docker to dockerCompatible.

Version 2018-03-01.6.1

New features in version 2018-03-01.6.1 include:

  • Pool node counts by state: This version adds the ability to query pool node counts by state, via the new ListPoolNodeCounts operation. This operation gives you the ability to query all pools in a Batch account for node states including creating, idle, offline, preempted, rebooting, reimaging, starting, and others.
  • Node agent logs: This version adds the ability to upload Azure Batch node agent logs from a particular node, via the UploadBatchServiceLogs operation. This is intended for use in debugging by Microsoft Support if problems occur on a node.

Version 2017-09-01.6.0

  • Azure Hybrid Use Benefit: You can now create Batch Windows VM pools specifying that Azure Hybrid Use Benefit licensing be used. When this licensing is used, a discount is applied to the VM price. Use the new LicenseType property on VirtualMachineConfiguration.

  • Pool VM data disks: You can attach one or more empty data disks to VM pools by using the new data disk configuration that is part of the virtual machine configuration. Use the DataDisks property on VirtualMachineConfiguration.

  • (Breaking change) Custom images are now created and deployed using Azure Resource Manager image resources instead of blob VHD files. Batch now uses managed disks to create the pool VMs and therefore uses an Image resource. There is now no limit to the number of VMs in a pool created using a custom image; also, multiple copies of the VHD blob for large pools are no longer required. Custom image pools are therefore much easier to create and scale. For more information, see Use a custom image to create a pool of virtual machines.

  • (Breaking change) Multi-instance tasks (created using MultiInstanceSettings) must now specify CoordinationCommandLine, and NumberOfInstances is now optional and defaults to 1.

  • Added support for tasks run using Docker containers. To run a task using a Docker container, you must specify a ContainerConfiguration on the VirtualMachineConfiguration for a pool, and then add TaskContainerSettings on the Task.

Version 2017-06-01.5.1

The version of the Batch API described here is 2017-06-01.5.1. Using the latest version is recommended when possible.

New features in version 2017-06-01.5.1 include:

  • Support for efficient task counts via the new Get Task Counts operation. By calling the Get Task Counts operation, you can get a count of active, running, and completed tasks, and of tasks that have succeeded or failed. For more information, see Count tasks for a job by state (Preview).
  • Support for specifying inbound endpoints on pool compute nodes, via the new poolEndpointConfiguration property. By setting this property, you can address specific ports on the node externally.

Version 2017-05-01.5.0

The following sections outline new and changed features in version 2017-05-01.5.0.

Important

Version 2017-05-01.5.0 includes several breaking changes. Please review the breaking changes noted below and update your code in order to call version 2017-05-01.5.0.

Low-priority compute nodes

Azure Batch now offers low-priority compute nodes to reduce the cost of Batch workloads. Low-priority VMs make new types of Batch workloads possible by providing a large amount of compute power that is also economical.

There are several changes to the REST API for low-priority nodes:

  • (Breaking change) The targetDedicated and currentDedicated properties of a pool or pool specification have been renamed to targetDedicatedNodes and currentComputeNodes.
  • (Breaking change) The resizeError property of a pool is now a collection called resizeErrors.
  • Compute nodes have a new isDedicated property. This property is true for dedicated nodes, and false for low-priority nodes.
  • A Job Manager task has a new allowLowPriorityNode property. If this property is true, the Job Manager task may run on a low-priority node. If it is false, then the Job Manager task will run on a dedicated compute node.

Named users on Linux

Azure Batch now offers additional support for creating named users on Linux nodes.

  • The new linuxUserConfiguration property supports specifying a uid (user ID) and gid (group ID) when creating a Linux user.
  • (Breaking change) The sshPrivateKey property has been moved from the userAccount property to the new linuxUserConfiguration property. The linuxUserConfiguration property is itself a property of the userAccount property.

For more information about named user accounts, see Run tasks under user accounts in Batch.

Output files for task data

You can now specify output files to upload task data after a task has completed.

  • The new outputFiles property supports specifying task files to upload to Azure Storage.
  • The new fileUploadError property of the exitConditions property supports specifying actions to take based on the upload status of an output file.

Task error reporting

Several changes have been made to improve reporting for task errors.

  • The new result property of the executionInfo property indicates whether a task succeeded or failed.
  • (Breaking change) The schedulingError property of the executionInfo property has been renamed failureInfo. The failureInfo property is returned any time there is a task failure. This includes all previous scheduling error cases, all cases where a nonzero task exit code is returned, and any file upload failures.
  • (Breaking change) The schedulingError property of the exitConditions property has been renamed to preProcessingError to clarify when the error took place in the task lifecycle.
  • (Breaking change) The schedulingErrorCateogry property has been renamed to errorCategory.

Application licenses

You can now request that application licenses be provisioned to your pool, via the new applicationLicenses property on a pool or pool specification.

Version 2017-01-01.4.0

This version release extends all support from the previous version, 2016-07-01.3.1. Additionally, it supports the following capabilities:

  • Run a task under a specified user identity.

    You can now run a task or task collection under one of the following user identities, specified via the new userIdentity property on the task resource:

    • A user account with a name that you define.

    • A user account that is created automatically (an auto-user). An auto-user can run as an administrative user or as a non-administrative user. By default, an auto-user runs as a non-administrative user.

      Important

      The userIdentity property, with its elevationLevel property, replaces the runElevated property in requests that add a task or a task collection, and in responses that get information about a task or that list tasks.

      If you make a request that includes the runElevated property to version 2017-01-01.4.0 of the Batch service, the request will fail.

      To run as an administrative user, update your application to use the userIdentity property, setting the elevationLevel property to admin.

      To run as a non-administrative user, update your application to use the userIdentity property, setting the elevationLevel property to nonAdmin. Since this is the default, you can also omit the setting.

  • Define user accounts across all nodes in a pool.

    You can now run a task or task collection under a user account that you define on the pool resource. Define a user account via the new userAccounts property in requests to Add Pool. When you define the account, you can specify the account name, password, elevation level (admin or non-admin), and SSH private key (for Linux pools).

    Once you define the user account, you can specify that user account for the userIdentity property in requests that add a task or a task collection.

  • Provide a task with a token to authenticate to the Batch service when the task runs.

    The Batch service can now provide an authentication token to a task when it runs. The authentication token enables a task to issue requests related to the job to the Batch service, without the Batch account keys. The token is provided via the AZ_BATCH_AUTHENTICATION_TOKEN environment variable.

    Currently authentication tokens are supported for calling operations on the job resource only. The authentication token grants access to all operations on the job that contains the task.

    To have the Batch service provide the authentication token, specify the new authenticationTokenSettings property, together with its access property, in requests to Add Task or Add Task Collection.

  • Specify an action to take on a task's dependencies if the task fails.

    You can now specify that dependent tasks proceed even if the task that they depend on fails. Set the new dependencyAction property of a task resource to satisfy to run dependent tasks even if the parent task fails. Alternately, set dependencyAction to block to prevent running of dependent tasks if the parent task fails.

    Specify the dependencyAction property in requests to Add Task or Add Task Collection.

    For more information about using dependency actions, see Dependency actions.

  • Use custom OS disk images when creating a pool.

    You can now use custom OS disk images to create a pool.

    To do so, you must specify when you create your Batch account that pools are to be provisioned in the user subscription, rather than in a subscription managed by the Batch service. In a call to Create Account, set the poolAllocationMode property to UserSubscription. Then use the osDisk property to specify a reference to a disk image in a request to Add Pool.

    Important

    When you create your Batch account, if you specify that pools are to be provisioned in the user subscription, then you must use Azure Active Directory-based authentication for all requests made through that account.

  • Use Azure Active Directory-based authentication for requests to the Batch service.

    Azure Active Directory (Azure AD) is now supported for authenticating calls to the Batch service.

    If your Batch account is set up to provision pools in the user subscription, then using Azure AD authentication is required.

Version 2016-07-01.3.1

This version release extends all support from previous version, 2016-02-01.3.0. Additionally, it supports the following capabilities:

  • Capability to create a pool and an autopool with Network configuration

    • A new property networkConfiguration has been added to both the pool and autopool resources. This property can be used to specify the pool's network configuration, such as the subnet in which the pool's compute nodes is created.

  • Automatically terminate a job when all tasks are complete.

    • A new property onAllTasksComplete has been added to the job resource. You can specify this when you create or update a job.

  • Automatically terminate or disable a job when a task fails.

    • A new property onTaskFailure has been added to the job resource, and a new property exitConditions has been added to the task resource. The job onTaskFailure setting indicates whether task failure may have a job-level impact, and the task exitConditions property allows tasks to tailor the job-level impact to the nature of the failure.

  • Associate application packages with tasks as well as pools.

    • A new property applicationPackageReferences has been added to the task resource and to the job jobManagerTask settings. This allows application packages to be deployed on demand as tasks require them rather than requiring them to be specified at pool creation time. This is particularly valuable for long-lived pools that may be called on to run different kinds of jobs or different application versions over their lifetime, or if you have a large application package that is not required by all tasks.

  • Capability to reactivate a failed task.

    • A new operation Reactivate a task has been added to reset a failed task's state to active. This allows failures to be retried, for example if the failure was transient or if you have been able to fix the cause of the failure, without recreating the task.

Version 2016-02-01.3.0

This version release extends all support from previous version, 2015-12-01.2.2. Additionally, it supports the following capabilities:

Version 2015-12-01.2.2

This version release extends all support from previous version, 2015-11-01.2.1. Additionally, it supports the following capabilities:

  • Applications can now be deployed to compute nodes using application packages instead of as resource files.

    • New APIs are provided for clients to browse the list of available applications and versions, for example to display a selector in a user interface.

    • Pool-related APIs are changed to allow a pool to specify application packages to be downloaded to all nodes in that pool.

  • Tasks can now be made dependent on other tasks. A dependent task will not start until all the tasks it depends on have completed successfully.

This version release includes the following other changes:

  • The default pool resize timeout has changed to 15 minutes.

Version 2015-11-01.2.1

This version release extends all support from previous version 2015-06-01.2.0. Additionally, it supports the following capabilities:

  • Capability to add and run multi-instance task (for example, MPI)

    • A new optional property ‘multiInstanceSettings’ is added to the Task resource that can be optionally included in Add Task, Get Task, List Tasks APIs.

    • A new API List the subtasks of a task is added to obtain information about subtasks.

    • A new optional property ‘subtaskId’ is added to ‘recentTasks’ for a Compute Node. This property is returned as part of Get Compute Node and List Compute Nodes

  • Capability to disable or enable task scheduling on a compute node

    • Two new APIs Disable task scheduling on a node and Enable task scheduling on a node are added on compute node resource.

    • A new compute node state ‘offline’ is added. For older API versions, this state is mapped to ‘unusable’.

    • A new property ‘schedulingState’ is added to the compute node resource. Possible values for ‘enabled’ and ‘disabled’.

  • Capability to specify autoscale evaluation interval on a pool/autopool

    • A new property ‘autoScaleEvaluationInterval’ is added to pool resource. This property can be included in Add Pool, Get Pool, Enable AutoScale, and List Pools APIs.

    • Similarly, for auto pools, a new property ‘autoScaleEvaluationInterval’ is added to ‘autoPoolSpecification’ property. This property can be included in Add Job, Get Job, List Jobs, Add Job Schedule, Get Job Schedule, Update Job Schedule, List Job Schedules, and List Jobs in Job Schedule APIs.

  • Enhancement to Task APIs to return the root directory of the task on the compute node.

    • Two new properties ‘taskRootDirectory’ and ‘taskRootDirectoryUrl’ are added to Task resource that can be obtained as part of Get information about a task, List Tasks APIs

    • A new property ‘taskRootDirectory’ is added to ‘jobPreparationTaskExecutionInfo’ and ‘jobReleaseTaskExecutionInfo’ which can be obtained via List the status of the job preparation and job release tasks for a job API.