Can't setting the scoring script when creating batch endpoint job

Question

Can't setting the scoring script when creating batch endpoint job

Yicong Cui 20 Microsoft Employee

I have a deployment in the Azure Machine Learning batch endpoint. In this deployment, I set a custom scoring script batch_driver.py. (see below picture)

azure_ml_job

However, when I try creating new job, the scoring name still is driver/amlbi_main.py. Is it a bug?

azure_ml_job2

Amira Bedhiafi 41,721 Reputation points MVP Volunteer Moderator

2025-03-07T13:13:02.4333333+00:00

Can you share more information? The pictures are not visible
Manas Mohanty 16,840 Reputation points Microsoft External Staff Moderator

2025-03-10T07:58:27.7966667+00:00

Hi Yicong Cui

We are checking to see if the above pointers helped address your issue.

Thank you.

Yicong Cui 20 Microsoft Employee

Hi Manas Mohanty. After checking the logs, I found an issue caused in the runtime. I fixed it but still blocking by some authentication issue.

In my endpoint deployment, I'm using a custom environment. Below is the yaml file of my batch endpoint deployment. I specify the script but, in the job, the script name is driver/amlbi_main.py. Is it expected or not?

$schema: https://azuremlschemas.azureedge.net/latest/modelBatchDeployment.schema.json
endpoint_name: endpoint-uunyz
name: npe-batch-endpoint
description: Batch endpoint 
type: model
model: azureml:npe_decision_tree_model@latest
compute: azureml:NPEComputeCluster
environment: azureml:model-training:22
code_configuration:
  code: ../../../
  scoring_script: batch_driver.py
resources:
  instance_count: 1
settings:
  max_concurrency_per_instance: 2
  mini_batch_size: 2
  output_action: append_row
  output_file_name: predictions.csv
  retry_settings:
    max_retries: 3
    timeout: 300
  error_threshold: -1
  logging_level: info

Manas Mohanty 16,840 Reputation points Microsoft External Staff Moderator

2025-03-10T09:03:38.2333333+00:00

Hi Yicong Cui

You can check the authentication section to use different type of credentials like service principal, managed identity etc.

During runtime, any kind of scoring script issue arise, it might be due to missing package, loading models.

driver/amlbi_main.py - As per my past encounters, it points overall towards runtime issue mostly coming from multiple reason which might be causing failure in scoring the inputs.

We should check multiple sources of logs endpoint logs, environment logs, scoring script logs to find the blocking elements.

Thank you
Yicong Cui 20 Reputation points Microsoft Employee

2025-03-10T09:16:21.55+00:00

Regarding the authentication problem, I checked the permission according to this doc but still failed. Finally, I solved it by adding the role Storage Table Data Contributor for the identity.
Yicong Cui 20 Reputation points Microsoft Employee

2025-03-10T09:18:09.22+00:00

Regarding to the authentication issue, I have some tries according to this doc but it still failed.

Finally, I try adding the Storage Table Data Contributor role for my identities and it works.

Answer accepted by question author

0 additional answers

Your answer

Amira Bedhiafi 41,721 Reputation points MVP Volunteer Moderator

2025-03-07T13:13:02.4333333+00:00

Can you share more information? The pictures are not visible
Manas Mohanty 16,840 Reputation points Microsoft External Staff Moderator

2025-03-10T07:58:27.7966667+00:00

Hi Yicong Cui

We are checking to see if the above pointers helped address your issue.

Thank you.
Yicong Cui 20 Reputation points Microsoft Employee

2025-03-10T08:07:35.0733333+00:00

Hi Manas Mohanty. After checking the logs, I found an issue caused in the runtime. I fixed it but still blocking by some authentication issue.

In my endpoint deployment, I'm using a custom environment. Below is the yaml file of my batch endpoint deployment. I specify the script but, in the job, the script name is driver/amlbi_main.py. Is it expected or not?

$schema: https://azuremlschemas.azureedge.net/latest/modelBatchDeployment.schema.json endpoint_name: endpoint-uunyz name: npe-batch-endpoint description: Batch endpoint type: model model: azureml:npe_decision_tree_model@latest compute: azureml:NPEComputeCluster environment: azureml:model-training:22 code_configuration: code: ../../../ scoring_script: batch_driver.py resources: instance_count: 1 settings: max_concurrency_per_instance: 2 mini_batch_size: 2 output_action: append_row output_file_name: predictions.csv retry_settings: max_retries: 3 timeout: 300 error_threshold: -1 logging_level: info
Manas Mohanty 16,840 Reputation points Microsoft External Staff Moderator

2025-03-10T09:03:38.2333333+00:00

Hi Yicong Cui

You can check the authentication section to use different type of credentials like service principal, managed identity etc.

During runtime, any kind of scoring script issue arise, it might be due to missing package, loading models.

driver/amlbi_main.py - As per my past encounters, it points overall towards runtime issue mostly coming from multiple reason which might be causing failure in scoring the inputs.

We should check multiple sources of logs endpoint logs, environment logs, scoring script logs to find the blocking elements.

Thank you
Yicong Cui 20 Reputation points Microsoft Employee

2025-03-10T09:16:21.55+00:00

Regarding the authentication problem, I checked the permission according to this doc but still failed. Finally, I solved it by adding the role Storage Table Data Contributor for the identity.
Yicong Cui 20 Reputation points Microsoft Employee

2025-03-10T09:18:09.22+00:00

Regarding to the authentication issue, I have some tries according to this doc but it still failed.

Finally, I try adding the Storage Table Data Contributor role for my identities and it works.

Answer 1

Manas Mohanty 16,840 Microsoft External Staff Moderator

Hi Yicong Cui

Issue seems to be in scoring script. Normally driver/amlbi_main.py issues point towards missing environment packages or runtime issue in loading model etc.

You can debug it by adding try catch blocks in crucial steps like model/data loading, saving to storage location etc.

First point of debugging should be in endpoint logs.

If you could share endpoint and environment logs in private message, we can debug further.

Attached Troubleshoot guide for batch endpoint for reference

Thank you.

Manas Mohanty 16,840 Reputation points Microsoft External Staff Moderator

2025-03-11T09:40:57.1933333+00:00

Hi Yicong Cui

Could you upvote the above answer if it helped.

Thank you.

Share via

Can't setting the scoring script when creating batch endpoint job

0 additional answers

Your answer