Microsoft Azure Docker Error

Philip M 1

Hello,

last week when I ran experiments on azure (compute is NC6 (GPU)) everything worked fine. Now I get the error:

docker: Error response from daemon: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"process_linux.go:432: running prestart hook 1 caused \\"error running hook: exit status 1, stdout: , stderr: nvidia-container-cli: mount error: file creation failed: /opt/docker/overlay2/522acf94adc315a38d74b55de4cb84f9b695b0073de8a3f13da13d8ab0302209/merged/run/nvidia-persistenced/socket: no such device or address\\n\\"\"": unknown.

after that it tries to restart docker a couple of times and displays the following error message each time until it aborts:

Retry...
2020-08-12T09:36:14Z Force Restart Docker Service
2020-08-12T09:36:16Z
2020-08-12T09:36:16Z docker: Error response from daemon: Conflict. The container name "/bert_1597224769_6fd177b7" is already in use by container "0df086c7601f1b6908f95d064d4251e5edaae80220cdffa747784a16bb740b87". You have to remove (or rename) that container to be able to reuse that name.

Could you please tell me how to fix this issue?

Regards,

Philip

prmanhas-MSFT 17,906 Reputation points Microsoft Employee

2020-08-12T12:07:35.377+00:00

@Philip M Thank you for your question.

As per the error what I can see is that there is some discrepancy with container names. Can you please confirm if you have already tried renaming or removing the container?
Philip M 1 Reputation point

2020-08-13T19:34:48.937+00:00

Yes, when I give my container another name, even one I haven't used before, then the error message just says that the other name "is already in use".
When I run experiments on azure, I create an Estimator and the container name before _ (in this case "bert") is the name of my experiment. To see if I still get the error if I start from scratch, I created a new resource group with workspace and compute, but I got this error as well.
prmanhas-MSFT 17,906 Reputation points Microsoft Employee

2020-08-17T17:29:00.933+00:00

@Philip M Thank you for your response and your patience over the matter.

I have reached out to our internal team on this.Once I have an update,will keep you posted on same.

Thanks
prmanhas-MSFT 17,906 Reputation points Microsoft Employee

2020-08-21T09:45:59.363+00:00

@Philip M Firstly, apologies for the delay in responding on this and any inconvenience this issue may have caused. As this issues need more investigation and live troubleshooting for quicker resolution I would recommend you to contact azure support.

If you have a support plan, requesting you to file a support ticket, else please do let us know, we will try and help you get a one-time free technical support.

In this case, could you send an email to AzCommunity@microsoft.com referencing this thread as well as your subscription ID. Please mention "ATTN: Preeti" in the subject line. Thank you for your cooperation on this matter and look forward to your reply.

Also, once you get the issue fixed, request you to reply back here on the thread with the resolution steps for the benefit of the community.

Share via

Microsoft Azure Docker Error

Your answer