Need to import a custom Python Module for a batch activity. How to mention it in start task.

Question

Hi,

I have a python Module say xyz (user defined, not default modules like pandas). I want to import this module for one of our batch activities. I want to understand the syntax to import it in pool settings.

1. Where do i save the zip file of the module, on any container in blob storage? (or on the same container that the python file is uploaded and called from custom activity)
2. How do i mention the path of this zip in batch pool settings? (in start task or support files or where)?

My folder structure:
myfolder_myutility

• _init.py
• Mypythonfile.py

In my python file thats called from batch activity, i have this line: from myfolder_myutility.mymodule import module

When i run batch activity, it is throwing, myfolder_myutility is not found,.

• please help.

Answer 1

Hello @Subashri Vasudevan and welcome to Microsoft Q&A.

As I understand it, you are looking for specific examples or directions on how to provide or locate libraries and other preperatory files for use in ADF Custom activity / Azure Batch.

The documentation has an example here.

I am calling out points labled in image below.

1. The "Resource Linked Service" allows you to specify a storage account. The account can be used to not only store output of the activity, but also to supply custom libraries and programs.
2. The "Folder Path" specifies a location of "Resoruce Linked Service" to upload to the batch and make available code or libraries or other assets for use. The "Resource Linked Service" points to the account in general, rather than a specific container/folder. The "Folder path" starts with selecting a container, then folder underneath. myContainer/myFolder/mySubfolder Use the browse option to ensure you got it right. The "browse local" lets you upload a folder from your local computer.
3. When the "Resouce Linked Service" and "Folder Path" are in use, the contents are made available for use in the command, as if it was your working directory.

So, in general, put the library in the same, or child location as your program. Placing them in different containers would not help.

Does this help?