Azure machine learning compute local

Question

Azure machine learning compute local

Pablo Díaz 16

Hello all, I have followed the https://learn.microsoft.com/en-us/azure/machine-learning/tutorial-azure-ml-in-a-day tutorial with successful result. Now, instead of using the compute cluster, i would like to use the resources of my local computer to run the job. I change the compute option to 'local' in the command function, but throws me an error: JobException: Failed to read in local executable job. Do i need additional configuration? Thanks in advance.

ashic 1

I'm using the v2 cli, and I have exactly the same issue. My job config looks like:

$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json  
code: ../src  
command: >-  
  python main.py   
  --diabetes-csv ${{inputs.diabetes_csv}}  
  --model_output ${{outputs.model_output}}  
inputs:  
  diabetes_csv:   
    type: uri_file  
    path: ../data/diabetes.csv  
outputs:  
  model_output:   
    type: uri_folder  
    path: ../output  
environment: azureml:AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest  
compute: local  
display_name: diabetes-training-demo  
experiment_name: diabetes-training-demo  
description: Train a scikit-learn LinearRegression model on the Diabetes dataset.

I get the same error: Failed to read in local executable job.

Ramr-msft 17,826

@Pablo Díaz Thanks for the question. Can you please share the code that you are trying for submitting the local run.
Here is the code to submit local run using SDK v2.
#import required libraries
from azure.ai.ml import MLClient, command
from azure.ai.ml.entities import Environment
from azure.identity import DefaultAzureCredential

#connect to the workspace  
ml_client = MLClient.from_config(DefaultAzureCredential())  
  
# set up pytorch environment  
env = Environment(  
    image='mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04',  
    conda_file='pytorch-env.yml',  
    name='pytorch-env'  
)  
  
# define the command  
command_job = command(  
    code='./src',  
    command='train.py',  
    environment=env,  
    compute='local',  
)  
  
returned_job = ml_client.jobs.create_or_update(command_job)  
returned_job

Pablo Díaz 16 Reputation points

2022-11-29T10:21:34.52+00:00

@Ramr-msft Good morning!! Thank you for your reply. I tried the code you wrote above but everytime i change parameters i get the same error: 'JobException: Failed to read in local executable job. If i create a compute, then set it in the command function it runs successfully and create the job in aml studio. I upload some pictures of the code and the error:

Any idea why this error is happening?

6 answers

Your answer

ashic 1 Reputation point

2022-11-29T01:12:35.237+00:00

I'm using the v2 cli, and I have exactly the same issue. My job config looks like:

$schema: https://azuremlschemas.azureedge.net/latest/commandJob.schema.json code: ../src command: >- python main.py --diabetes-csv ${{inputs.diabetes_csv}} --model_output ${{outputs.model_output}} inputs: diabetes_csv: type: uri_file path: ../data/diabetes.csv outputs: model_output: type: uri_folder path: ../output environment: azureml:AzureML-sklearn-1.0-ubuntu20.04-py38-cpu@latest compute: local display_name: diabetes-training-demo experiment_name: diabetes-training-demo description: Train a scikit-learn LinearRegression model on the Diabetes dataset.

I get the same error: Failed to read in local executable job.
Ramr-msft 17,826 Reputation points

2022-11-29T05:17:32.423+00:00

@Pablo Díaz Thanks for the question. Can you please share the code that you are trying for submitting the local run.
Here is the code to submit local run using SDK v2.
#import required libraries
from azure.ai.ml import MLClient, command
from azure.ai.ml.entities import Environment
from azure.identity import DefaultAzureCredential

#connect to the workspace ml_client = MLClient.from_config(DefaultAzureCredential()) # set up pytorch environment env = Environment( image='mcr.microsoft.com/azureml/openmpi3.1.2-ubuntu18.04', conda_file='pytorch-env.yml', name='pytorch-env' ) # define the command command_job = command( code='./src', command='train.py', environment=env, compute='local', ) returned_job = ml_client.jobs.create_or_update(command_job) returned_job
Pablo Díaz 16 Reputation points

2022-11-29T10:21:34.52+00:00

@Ramr-msft Good morning!! Thank you for your reply. I tried the code you wrote above but everytime i change parameters i get the same error: 'JobException: Failed to read in local executable job. If i create a compute, then set it in the command function it runs successfully and create the job in aml studio. I upload some pictures of the code and the error:

Any idea why this error is happening?

Answer 1

lennart van de Guchte 6

Any updates on this issue?

Answer 2

Pablo Díaz 16

@Ramr-msft Good morning!! Thank you for your reply. I tried the code you wrote above but everytime i change parameters i get the same error: 'JobException: Failed to read in local executable job. If i create a compute, then set it in the command function it runs successfully and create the job in aml studio. I upload some pictures of the code and the error:

Any idea why this error is happening?

Answer 3

Mark 26

I'm having the exact same problem with the example posted, I hope this is fixed soon.

Answer 4

Ramr-msft 17,826

@Mark @Pablo Díaz Thanks for the details. This is known issue and team is working on the same to fix the problem. I will update you once it's fixed and rolled out.

Answer 5

Amr Sharaf 1 Microsoft Employee

any updates? I still can't seem to be able to run locally using sdk v2, seems relevant to this:

https://github.com/Azure/azure-sdk-for-python/issues/28139

Jon-Paul Boyd 31 Reputation points

2023-03-03T11:14:25.9066667+00:00

Same issue here trying to use local compute for training job SDK v2

Share via

Azure machine learning compute local

6 answers

Your answer