Unknown error occurred during authentication. Error detail: Unexpected polling state code_expired

Lu 66 Reputation points
2021-06-30T15:16:22.16+00:00

Hi,
I want to submit some training scripts and got this error. In the meantime I got the warning:

{
  "error": {
    "code": "UserError",
    "severity": null,
    "message": "AzureMLCompute job failed.\nJobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details.\n\tReason: Job failed with non-zero exit Code",
    "messageFormat": "{Message}",
    "messageParameters": {
      "Message": "AzureMLCompute job failed.\nJobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details.\n\tReason: Job failed with non-zero exit Code"
    },
    "referenceCode": null,
    "detailsUri": null,
    "target": null,
    "details": [],
    "innerError": {
      "code": "UserTrainingScriptFailed",
      "innerError": null
    },
    "debugInfo": null,
    "additionalInfo": null
  },
  "correlation": {
    "operation": "4ce06386a423624f8db9086f4cde3909",
    "request": "96625a89e839edc7"
  },
  "environment": "westeurope",
  "location": "westeurope",
  "time": "2021-06-30T14:00:14.3853285+00:00",
  "componentName": "execution-worker"
}

I run the same scripts in a compute instance and didn't get the error above. Only if I submit it to the computer cluster I get this error.
Running in the compute instance I got the following failure messages, though the codes can run till the end:

Failure while loading azureml_run_type_providers. Failed to load entrypoint hyperdrive = azureml.train.hyperdrive:HyperDriveRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.PipelineRun = azureml.pipeline.core.run:PipelineRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.ReusedStepRun = azureml.pipeline.core.run:StepRun._from_reused_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.StepRun = azureml.pipeline.core.run:StepRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.scriptrun = azureml.core.script_run:ScriptRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0')).
Failure while loading azureml_run_type_providers. Failed to load entrypoint hyperdrive = azureml.train.hyperdrive:HyperDriveRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.PipelineRun = azureml.pipeline.core.run:PipelineRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.ReusedStepRun = azureml.pipeline.core.run:StepRun._from_reused_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.StepRun = azureml.pipeline.core.run:StepRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.scriptrun = azureml.core.script_run:ScriptRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0')).

Update: I changed to another conda environment and submit the scripts, still got the same error. Running in the terminal worked without any error and also without the failure messages.

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,718 questions
0 comments No comments
{count} votes

1 answer

Sort by: Most helpful
  1. YutongTie-MSFT 48,506 Reputation points
    2021-07-05T21:29:47.803+00:00

    The workaround from the customer for this issue:

    Remove the following lines for reading input data from main.py script, which is called in ScriptRunConfig, while submitting:

    ws = Workspace.from_config()
    datastore = ws.get_default_datastore()
    dataset = Dataset.File.from_files(path=(datastore, 'datasets/exampledata'))
    

    Thanks to Lu for sharing this.

    Regards,
    Yutong

    0 comments No comments