Failure in Sidecar: Failed to read file line by line.

schwarze 6 Reputation points
2021-04-29T05:47:53.623+00:00

I was running 100 identical jobs on a compute cluster via Azure Machine Learning of which 97 failed and 3 succeeded without issues. The failed jobs failed half-way through with the following error message

AzureMLCompute job failed.
JobFailed: Submitted script failed with a non-zero exit code; see the driver log file for details.
    Reason: Job failed with non-zero exit Code
    Reason: runSpecialJobTask failed with with non zero exit code
    stderr: runSpecialJobTask: postprocessing: Failure in Sidecar:
Failed to read file line by line.
    Reason: Job release task failed with non-zero exit Code

There is no error message in the driver log or any other log file to indicate what went wrong. It seems like the failed jobs stopped half-way for no reason. The wording of the error message "Failed to read file line by line." does not seem to be a standard python error message. Is there a way I can find out what happened in the sidecar and trace this error?

Azure Machine Learning
Azure Machine Learning
An Azure machine learning service for building and deploying models.
2,552 questions