Issue with Azure IoT Hub during Module Deployment

Question

Issue with Azure IoT Hub during Module Deployment

Javier 40

Hello,

I have been experiencing some issues with Azure IoT Hub when deploying updates to my module. My IoT Edge device has been working well for over a month, but recently I've started encountering errors.

The specific error message is:

Set Modules "A problem occurred attempting to set device modules on device <device id>. The request failed due to an issue with the IoT hub. Details: ErrorCode: ServerError;InternalServerError Error: Tracking ID: <some id> TimeStamp: 10/02/2024 15:06:05"

Could you please confirm if this issue is related to an internal problem with IoT Hub, or if there is something on my end that I need to address?

Thank you for your assistance.

Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-02T15:44:24.6366667+00:00

Hello @Javier

welcome to this moderated Azure community forum.

Can you share a bit more information?

Readers would like to know what AIE version you use on what OS.

Are the EdgeAgent and EdgeHub working as expected?

Do you encounter this also with other modules or previous versions of your module?

Are you able to read the recent logging of the module?

Did you check the edge CA certificate (for securing inter-module communication)?
Javier 40 Reputation points

2024-10-02T17:33:05.4533333+00:00

Hi Sander,

Here's the information based on my setup and current configuration:

Azure IoT Edge Version: I am using Azure IoT Edge version 1.5 on Ubuntu 22.04.

EdgeAgent and EdgeHub Status: Yes, both the EdgeAgent and EdgeHub modules are running as expected and have been stable for the past 2 weeks. Their status is reported as "running" when checked using the command sudo iotedge list.

Issue with Other Modules or Previous Versions: The issue primarily occurs with the custom module that is being deployed. This specific error (InternalServerError) has not been observed with other modules in the past or with previous versions of this module.

Recent Module Logging: Yes, I can access and review the recent logs of the modules using sudo iotedge logs <device>. There is consistent logging indicating proper startup and connectivity, but the InternalServerError occurs during the module deployment phase from the IoT Hub (in Azure Portal).

I also checked the logs for modules $edgeAgent and $edgeHub in the IoT Hub (in Azure Portal), and they look good.

Edge CA Certificate for Inter-Module Communication: I am using just one custom module. The IoT Hub was created 73 days ago. I have not manually configured any edge CA certificate. However, I see from the documentation that the default certificate validity is 90 days. I will review this configuration to ensure that the certificate hasn't expired and is valid for inter-module communication.

I believe the InternalServerError might indicate a transient issue with the IoT Hub module deployment service. It could also be related to a server-side problem within Azure. I am currently monitoring the status and will attempt redeployment in 1 hour..

Let me know if there’s more information you’d like to know or if specific logs would be helpful!

Best, Javier
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-02T18:55:07.1633333+00:00

Hello @Javier,

thanks for the clear update.

It all seems to point into the IoT Hub direction.

It could be some temporary issue. Otherwise, submitting an Azure support ticket would be the next step.

Regarding the certificate, this secures the communication between the edge hub module, custom modules, and optional child devices.

You can fix it by deploying a custom certificate, the config fix, or simply restarting the edge device (edge runtime?) for another 90 days of fun.
Javier 40 Reputation points

2024-10-03T00:39:31.5233333+00:00

Thank you Sander.

The issue was resolved, but I'm not certain whether it was due to restarting the IoT Edge Device or because of a temporary Azure IoT Hub outage.
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-03T09:58:13.5766667+00:00

Hello @Javier,

thanks for reporting back.

It's hard to tell indeed.

I recommend keeping an eye on this issue in case it reappears in the future.

1 answer

Your answer

Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-02T15:44:24.6366667+00:00

Hello @Javier

welcome to this moderated Azure community forum.

Can you share a bit more information?

Readers would like to know what AIE version you use on what OS.

Are the EdgeAgent and EdgeHub working as expected?

Do you encounter this also with other modules or previous versions of your module?

Are you able to read the recent logging of the module?

Did you check the edge CA certificate (for securing inter-module communication)?
Javier 40 Reputation points

2024-10-02T17:33:05.4533333+00:00

Hi Sander,

Here's the information based on my setup and current configuration:

Azure IoT Edge Version: I am using Azure IoT Edge version 1.5 on Ubuntu 22.04.

EdgeAgent and EdgeHub Status: Yes, both the EdgeAgent and EdgeHub modules are running as expected and have been stable for the past 2 weeks. Their status is reported as "running" when checked using the command sudo iotedge list.

Issue with Other Modules or Previous Versions: The issue primarily occurs with the custom module that is being deployed. This specific error (InternalServerError) has not been observed with other modules in the past or with previous versions of this module.

Recent Module Logging: Yes, I can access and review the recent logs of the modules using sudo iotedge logs <device>. There is consistent logging indicating proper startup and connectivity, but the InternalServerError occurs during the module deployment phase from the IoT Hub (in Azure Portal).

I also checked the logs for modules $edgeAgent and $edgeHub in the IoT Hub (in Azure Portal), and they look good.

Edge CA Certificate for Inter-Module Communication: I am using just one custom module. The IoT Hub was created 73 days ago. I have not manually configured any edge CA certificate. However, I see from the documentation that the default certificate validity is 90 days. I will review this configuration to ensure that the certificate hasn't expired and is valid for inter-module communication.

I believe the InternalServerError might indicate a transient issue with the IoT Hub module deployment service. It could also be related to a server-side problem within Azure. I am currently monitoring the status and will attempt redeployment in 1 hour..

Let me know if there’s more information you’d like to know or if specific logs would be helpful!

Best, Javier
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-02T18:55:07.1633333+00:00

Hello @Javier,

thanks for the clear update.

It all seems to point into the IoT Hub direction.

It could be some temporary issue. Otherwise, submitting an Azure support ticket would be the next step.

Regarding the certificate, this secures the communication between the edge hub module, custom modules, and optional child devices.

You can fix it by deploying a custom certificate, the config fix, or simply restarting the edge device (edge runtime?) for another 90 days of fun.
Javier 40 Reputation points

2024-10-03T00:39:31.5233333+00:00

Thank you Sander.

The issue was resolved, but I'm not certain whether it was due to restarting the IoT Edge Device or because of a temporary Azure IoT Hub outage.
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-03T09:58:13.5766667+00:00

Hello @Javier,

thanks for reporting back.

It's hard to tell indeed.

I recommend keeping an eye on this issue in case it reappears in the future.

Answer 1

Javier 40

I created a support ticket and Microsoft called me to confirm that the system was down. It was impacting our SLA for the IoT Hub.

Today I am getting another InternalServerError. I cannot delete devices:

Unable to delete devices. The request failed due to an issue with the IoT hub Details: ErrorCode:ServerError;InternalServerError Error: Tracking ID:1274872fed424376b64637a9e54ebabe-G:0-TimeStamp:10/04/2024 17:17:30

Lets hope they fix this issue soon.

LeelaRajeshSayana-MSFT 17,866 Reputation points Moderator

2024-10-04T21:37:54.3233333+00:00

Hi @Javier Thank you for reporting the issue back on the community. This helps other community members running into a similar issue know the cause. I appreciate it if you can share the support ticket reference number so we can track the issue internally.
Sander van de Velde | MVP 36,951 Reputation points MVP Volunteer Moderator

2024-10-07T22:09:50.9033333+00:00

Hello @Javier, Any update on this?
LeelaRajeshSayana-MSFT 17,866 Reputation points Moderator

2024-10-14T16:36:43.3033333+00:00

Hi @Javier I am circling back to you to see if there is any resolution shared with you on this issue.

Share via

Issue with Azure IoT Hub during Module Deployment

1 answer

Your answer