Unknown error occurred during authentication. Error detail: Unexpected polling state code_expired

Question

Hi,
I want to submit some training scripts and got this error. In the meantime I got the warning:

{
  "error": {
    "code": "UserError",
    "severity": null,
    "message": "AzureMLCompute job failed.\nJobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details.\n\tReason: Job failed with non-zero exit Code",
    "messageFormat": "{Message}",
    "messageParameters": {
      "Message": "AzureMLCompute job failed.\nJobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details.\n\tReason: Job failed with non-zero exit Code"
    },
    "referenceCode": null,
    "detailsUri": null,
    "target": null,
    "details": [],
    "innerError": {
      "code": "UserTrainingScriptFailed",
      "innerError": null
    },
    "debugInfo": null,
    "additionalInfo": null
  },
  "correlation": {
    "operation": "4ce06386a423624f8db9086f4cde3909",
    "request": "96625a89e839edc7"
  },
  "environment": "westeurope",
  "location": "westeurope",
  "time": "2021-06-30T14:00:14.3853285+00:00",
  "componentName": "execution-worker"
}

I run the same scripts in a compute instance and didn't get the error above. Only if I submit it to the computer cluster I get this error.
Running in the compute instance I got the following failure messages, though the codes can run till the end:

Failure while loading azureml_run_type_providers. Failed to load entrypoint hyperdrive = azureml.train.hyperdrive:HyperDriveRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.PipelineRun = azureml.pipeline.core.run:PipelineRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.ReusedStepRun = azureml.pipeline.core.run:StepRun._from_reused_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.StepRun = azureml.pipeline.core.run:StepRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.scriptrun = azureml.core.script_run:ScriptRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0')).
Failure while loading azureml_run_type_providers. Failed to load entrypoint hyperdrive = azureml.train.hyperdrive:HyperDriveRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint automl = azureml.train.automl.run:AutoMLRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.PipelineRun = azureml.pipeline.core.run:PipelineRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.ReusedStepRun = azureml.pipeline.core.run:StepRun._from_reused_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.StepRun = azureml.pipeline.core.run:StepRun._from_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0'), {'azureml-core'}).
Failure while loading azureml_run_type_providers. Failed to load entrypoint azureml.scriptrun = azureml.core.script_run:ScriptRun._from_run_dto with exception (azure-mgmt-storage 18.0.0 (/anaconda/envs/azureml_py38/lib/python3.8/site-packages), Requirement.parse('azure-mgmt-storage<16.0.0,>=1.5.0')).

Update: I changed to another conda environment and submit the scripts, still got the same error. Running in the terminal worked without any error and also without the failure messages.

Answer 1

The workaround from the customer for this issue:

Remove the following lines for reading input data from main.py script, which is called in ScriptRunConfig, while submitting:

ws = Workspace.from_config()
datastore = ws.get_default_datastore()
dataset = Dataset.File.from_files(path=(datastore, 'datasets/exampledata'))

Thanks to Lu for sharing this.

Regards,
Yutong