python dependencies in batch api

Question

Hello,

I am trying to run a python script that has some package dependencies beyond that of the standard libraries. Specifically, I need pandas and numpy. Im in a windows environment. Ive been hunting for hours and cant quite find it. Here is my pool creation for now, and it doesnt work. Am I just not going about it the right way? Do I need make an "application package" or something? That seems overkill if Im able to call the command line directly.

   task_commands = ["curl -fSsL https://bootstrap.pypa.io/get-pip.py | python", "pip install numpy", "pip install 
        pandas", "python myScript.py 1 3"]

    pool = batchmodels.PoolAddParameter(
        id=pool_id,
        virtual_machine_configuration=batchmodels.VirtualMachineConfiguration(
            image_reference=image_ref_to_use,
            node_agent_sku_id=sku_to_use),
        vm_size=vm_size,
        target_dedicated_nodes=vm_count,
        start_task=batchmodels.StartTask(
            command_line=common.helpers.wrap_commands_in_shell(windows,task_commands),
            resource_files=[batchmodels.ResourceFile(
                            file_path=task_name,
                            http_url=sas_url)]))

Answer 1

Hi,

I assume you try to run this start task command in standard windows image. However, the standard windows doesn't have python pre-installed. You have to install the python first. Also "curl" is not a standard windows command.

Thanks

Answer 2

Hi, I just changed the code a bit - it should work now

  task_commands = ["curl -fSsL https://bootstrap.pypa.io/get-pip.py | python", "pip install numpy", "pip install 
         pandas", "python myScript.py 1 3"]
     pool = batchmodels.PoolAddParameter(
         id=pool_id,
         virtual_machine_configuration=batchmodels.VirtualMachineConfiguration(
             image_reference=image_ref_to_use,
             node_agent_sku_id=sku_to_use),
         vm_size=vm_size,
         target_dedicated_nodes=vm_count,
         start_task=batchmodels.StartTask(
             command_line=common.helpers.wrap_commands_in_shell(windows,task_commands),
             resource_files=[batchmodels.ResourceFile(
                             file_path=task_name,
                             http_url=sas_url)]))

Plus os path expanduser method