question

Laimis-8077 avatar image
0 Votes"
Laimis-8077 asked ramr-msft commented

Train model fail / error

Hello, when I create a model and try to train it, it always fails. The same model on different two compute targets are these:

  1. AzureMLCompute job failed. UserProcessKilledBySystemSignal: Job failed since the user script received system termination signal usually due to out-of-memory or segfault. Reason: Process Killed with either 6:aborted or 9:killed or 11:segment fault. exit code here is from wrapping bash hence 128 + n Cause: killed TaskIndex: NodeIp: 10.0.0.4 NodeId: tvmps_2b4c1352eab879faa7df6dd68985461ea7ef172338311bc4bd278f1c7c66b3ad_d Reason: Job failed with non-zero exit Code

  2. AmlExceptionMessage:AzureMLCompute job failed. JobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details. Reason: Job failed with non-zero exit Code ModuleExceptionMessage:InvalidTrainingDataset: Dataset contains invalid data for training. Learner type: Binary classifier. Reason: The number of label classes should equal to 2, got 5 classes.

  3. AzureMLCompute job failed. JobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details. Reason: Job failed with non-zero exit Code


azure-machine-learning
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

1 Answer

ramr-msft avatar image
1 Vote"
ramr-msft answered ramr-msft commented

@Laimis-8077 Thanks for the question. Can you please add more details about the steps that you performed and compute cluster details to check.
Can you please confirm are you using the AML Studio Designer to train the model?
Is the AML storage account restricts access to specific VNETs and the Compute Cluster isn’t in that VNET?

Also please confirm did you change your Default storage account key?

You can Update storage account key with below command.
Change storage account access keys - Azure Machine Learning | Microsoft Docs
az ml workspace sync-keys -w myworkspace -g myresourcegroup


· 2
5 |1600 characters needed characters left characters exceeded

Up to 10 attachments (including images) can be used with a maximum of 3.0 MiB each and 30.0 MiB total.

Hello, thank you for your answer!
Well, I am begginer in here, so I will try my best to answer.
Yes, I am using Azure ML Studio Designer to train the model. Created the model and everything is OK until it reaches train model step. I used free subscirption, then changed into pay-as-you-go, so I am using what is provided.

I would be very thankful if you could help me step-by-step what I need to do or what I need to provide here to solve this problem.
Thank you in advance!

0 Votes 0 ·

@Laimis-8077 Thanks for the details. It's intermittent issue, we would recommend to raise a Azure support desk ticket from Help+Support blade from Azure portal. This will help you to share the details securely and work with an engineer who can provide more insights about the issue that if it can be replicated.

0 Votes 0 ·