Error when creating batch pool in azure batch, task doesn't finish in azure data factory

Question

Hello,

I try to set up a batch pool in order to run some python scripts that clean complex datasets orchestrated by azure data factory.
Upon creating a pool I get the following error: AccountVMSeriesCoreQuotaReached.

Ignoring the error and scheduling a task to run the python script in adf leads to the pipeline running for 1 hour without completion. (python script is stored in a blob container)

What am I possibly doing wrong?

Answer 1

Azure places limits on the amount of vCPUs you can allocate for each VM size. If your subscription does not have any cores, then your batch job will not be able to allocate. Here is a doc that outlines the process to request more cores for a given VM size.

The default quota limits that you are seeing are placed on your subscription when it is created, and sometimes modified depending upon VM availability. If you need more cores in a particular region and of a specific size, just request it using the process I linked above. Once the cores have been allocated, the batch job should run without an issue.

Answer 2

I am having the exact same issue here. I've checked the quotes in my batch account, and saw all but the Dv3-series have a value of 0. Selecting a Dv3-series in a pool does prevent the AccountVMSeriesCoreQuotaReached-error.

Why are all these quota values 0?

How can we use for example A series in a batch job?