Share via

Creating an Azure Batch Service Pool using an ADF pipeline

Vivek Nimmagadda 0 Reputation points
2023-07-18T04:35:19.3566667+00:00

I want to automatically create an Azure Batch Service pool when an ADF pipeline is run.

I created a web activity inside the pipeline and passed the parameters under settings as follows:

URL: https://management.azure.com/subscriptions/{Subscription ID}/resourceGroups/{Resource Group Name}/providers/Microsoft.Batch/batchAccounts/{Azure Batch Account Name}/pools?api-version=2023-05-01 Method: POST Body: { "id": "pool2", "vmSize": "standard_a1", "virtualMachineConfiguration": { "imageReference": { "publisher": "Canonical", "offer": "0001-com-ubuntu-server-focal", "sku": "20_04-lts", "version": "latest" }, "nodeAgentSKUId": "batch.node.ubuntu 20.04" }, "resizeTimeout": "PT15M", "targetDedicatedNodes": 5, "targetLowPriorityNodes": 0, "taskSlotsPerNode": 3, "taskSchedulingPolicy": { "nodeFillType": "spread" }, "enableAutoScale": false, "enableInterNodeCommunication": true, "metadata": [ { "name": "myproperty", "value": "myvalue" } ] } as per the following official documentation (https://learn.microsoft.com/en-us/rest/api/batchservice/pool/add?tabs=HTTP) Authentication: None Resource: https://learn.microsoft.com/en-us/rest/api/batchservice/pool/add? Headers: Content-Type --> application/json, Authorization --> Basic @{base64(concat('{Batch Account Name}', ':', '{Batch Account Primary Key}'))}

enter image description here

But I keep getting the following error:

[enter image description here

](https://i.stack.imgur.com/wqTJh.png)

These are the following Authentication types that are currently being displayed under settings:

enter image description here

I also tried removing the Authorization header, but I am getting the following error:

[enter image description here

](https://i.stack.imgur.com/0EICe.png)

I even tried changing the Authentication to 'System Assigned Managed Identity'. However, ADF says that resource doesn't exist which is weird because the same works for GET and DELETE methods respectively.

[enter image description here

](https://i.stack.imgur.com/UlmOI.png)

So not exactly sure how to provide the authorization header in the correct format or how to generate a refresh token for Azure Batch Account. Any help would be greatly appreciated. Please let me know if something is not clear.

It would have been helpful if the official documentation included an example to do the same.

Azure Data Factory
Azure Data Factory
An Azure service for ingesting, preparing, and transforming data at scale.
10,198 questions

1 answer

Sort by: Most helpful
Most helpful Newest Oldest
  1. ShaikMaheer-MSFT 38,401 Reputation points Microsoft Employee
    2023-07-19T08:49:04.76+00:00

    Hi

    Thank you for posting query in Microsoft Q&A Platform.

    To create batch pool, could you please consider using REST API documented in below link. Please note its a PUT request.

    https://learn.microsoft.com/en-us/rest/api/batchmanagement/pool/create?tabs=HTTP

    Also, consider getting access token first and then use that access token in your API call.

    Kindly consider testing your API first with some third party tool such as Postman and then try same in web activity.

    Hope this helps. Please let me know how it goes. Thank you.