Is a Azure Batch dedicated node always running?

Question

Hello,

We use Azure Batch for a very simple task in Azure Data Factory. Recently it occured to us that we have very high VM costs and we're trying to find out why. We know the Azure billing tool might give us more insights, but we're working with a company that is very slow in giving us the appropriate rights. To stop this as soon as possible, we have to find other ways and now we're guessing this might be an issue.

It is the first time that I worked with Azure Batch. It took me some trial and error, but eventually I got Azure Batch to work setting Target dedicated nodes to 1. Here I assumed that we only have to pay for the computing power of this single task, but now I'm afraid this is a wrong assumption and that the VM behind this node is always running and is giving us these high costs.

Am I right about this? And if so, what can I do best? I read something about auto-scale and I would guess that if you program it correctly it shoudl be able to turn on one node when the tasks starts and kills it afterwards.

I would really appreciate if somebody could help me as soon as possible. Thank you in advance!

Best regards

Answer 1

@Erp, Wessel van

Welcome to Microsoft Q&A Platform, thanks for posting your query here.

Am I right about this?

As long as a node is allocated into a pool, it incurring charges. This has nothing to do with the time the task is being executed. Compute time, is essentially the lifetime of the node in the pool. As long as it is allocated to the pool, it is incurring charges.

For Autoscale compute nodes, please check this document.

Hope that helps.
If the suggested response helped you resolve your issue, please 'Accept as answer', so that it can help others in the community looking for help on similar topics.

Answer 2

There are two types of pool, fixed and auto-scale. I assume that you use fixed pool and target VM # is 1. Then the VM will be always running and charge cost.
If you need run task occasionally and on-demand, you can try auto-scale pool, with auto-scale pool, you can setup initial VM # as 0, and do scale-out/in according to active task #.

You can refer to the document to configure your auto-scale formular:
https://learn.microsoft.com/zh-cn/azure/batch/batch-automatic-scaling