The costing for the Batch services and I have trouble running my Python job in Azure Batch because the required directories are not present on the Batch node.

Question

1. I would like to know how Batch service is charged.
2. When I run my Python jobs through ADF's Batch service, I get som troubles.

For example: My test.py script and the utils folder are located in the run_python folder. The test.py script calls the test1.py script in the utils folder. But when running the script using ADF, there is an issue where it cannot find the utils folder.

How I can keep the structure in Batch service's node.

Answer 1

Azure Batch service charges are based on the following components:

1. Compute costs: Charges for VMs you use in your pool and the cost depends on the VM size and the number of VMs.
2. Storage costs: Charges for storing files in Azure Storage, including input data, application packages, and output data.
3. Data transfer costs: Charges for data transfer between the Azure Batch service and Azure Storage, as well as between different regions.

Regarding your issue with running the Python job, it looks like the problem is with the directory structure itself and the way paths are referenced in your scripts.

You can create a zip file of your run_python folder, including both the test.py script and the utils folder. This will ensure that the directory structure is preserved when the files are uploaded to the Batch node.

Then, upload the zip file to an Azure Storage account that is accessible by your Azure Batch account.

In your Azure Batch task, include a startup script that will unzip the package in the correct location on the Batch node.