IoT Edge custom machine learning module reported error status

Question

IoT Edge custom machine learning module reported error status

Paolo Ranzani 6

Hi,

I have developed a ML solution based on this tutorial.
I adapted the Python notebook to run into an existing ML workspace.
Unfortunately, when I tried to create Docker image, it failed with error 500. I found out that the Image class is deprecated (suppose this was the reason of the error), so I relied on the Environment class (in particular this documentation) and everything worked fine.
Also, testing the model as Web Service ACI endpoint works correctly, providing the result:
['{"machine": {"temperature": 31.16469009, "pressure": 2.158002669}, "ambient": {"temperature": 21.17794693, "humidity": 25}, "timeCreated": "2017-10-27T18:14:02.4911177Z", "anomaly": false}']

The issue I need support is the following: after deploying the ML model as container to the Edge device (in this case a Ubuntu VM created using this template), the ML module reported an error:

No logs are displayed, neither through VM SSH access, nor through the Portal.

The only information that I found out is this one:

I checked multiple times the correctness of the image URI for the ML module, and I'm pretty confident that it is, since the ACI is based on it and the Web Service test was succeded.

What could be the problem here?

Thanks

Rohit Mungi 49,131 Reputation points Microsoft Employee Moderator

2021-11-17T10:54:36.03+00:00
@Paolo Ranzani Since the container seems to have restarted a lot of times you can set the restartPolicy to Never and try the deployment again check the logs of your container on the edge device to check what might be causing this.

iotedge logs <container name>

With the restart policy set to never you can try running the docker command manually on the device after it fails and check if any logs are displayed.
Paolo Ranzani 6 Reputation points

2021-11-17T16:56:03.687+00:00

Hi @Rohit Mungi , thanks.
Based on your suggestion, I set the "restartPolicy" to "Never" and deployed again. As you can see, now the Runtime Status has became "Stopped":

Unfortunately no logs are diplayed with the command above.

As Container Create Options, I let the default value, i.e.,
{
"WorkingDir": "/var/azureml-app"
}

Should I add something in your opinion?
Rohit Mungi 49,131 Reputation points Microsoft Employee Moderator

2021-11-18T08:57:06.753+00:00
Ok. I see the following createOptions in the Deployment.json in the linked tutorial.

"createOptions": "{\"HostConfig\":{\"Binds\":[\"/data/misc/camera:/app/vam_model_folder\"],\"NetworkMode\":\"host\"},\"NetworkingConfig\":{\"EndpointsConfig\":{\"host\":{}}},\"WorkingDir\":\"/var/azureml-app\"}"

Not sure, if any of the missing options are the reason.
Paolo Ranzani 6 Reputation points

2021-11-19T15:28:40.7+00:00

Tested with those options too, but nothing changes.