Can all communication logs between devices and the cloud be obtained from IoT Hub logs, associated with each device ID?

Question

Can all communication logs between devices and the cloud be obtained from IoT Hub logs, associated with each device ID?

KURAMOTOAHUJATAIRA-3118 250

Is it possible to retrieve all communication logs between devices and the cloud in IoT Hub's logs associated with device IDs? I want to count the number of communications between each device and the cloud.

I have tried the following query and confirmed that I can retrieve the Connect/Disconnect logs for the devices:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.DEVICES" and ResourceType == "IOTHUBS"
| where Category == "Connections"

However, when I tried to retrieve the logs for direct methods using the following query within the date range in which I sent the direct methods to the devices, no results were returned:

AzureDiagnostics
| where ResourceProvider == "MICROSOFT.DEVICES" and ResourceType == "IOTHUBS"
| where Category == "DirectMethods"

Could you please let me know if there is anything you can share on this matter? Thank you.

Sander van de Velde | MVP 36,781 Reputation points MVP Volunteer Moderator

2023-05-11T23:29:13.37+00:00
I check one of my IoTHub and it is working on my side:

using:

AzureDiagnostics | where ResourceProvider == "MICROSOFT.DEVICES" and ResourceType == "IOTHUBS" | where Category == "DirectMethods"
LeelaRajeshSayana-MSFT 17,771 Reputation points Moderator

2023-05-22T22:20:29.7133333+00:00

Hi @KURAMOTOAHUJATAIRA-3118 We still have not heard back from you. Just wanted to check if the answer provided below was helpful? If it answers your query, please do click Accept Answer and Yes for the answer, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know. Thank you

Accepted answer

0 additional answers

Your answer

Sander van de Velde | MVP 36,781 Reputation points MVP Volunteer Moderator

2023-05-11T23:29:13.37+00:00

I check one of my IoTHub and it is working on my side:

using:

AzureDiagnostics | where ResourceProvider == "MICROSOFT.DEVICES" and ResourceType == "IOTHUBS" | where Category == "DirectMethods"
LeelaRajeshSayana-MSFT 17,771 Reputation points Moderator

2023-05-22T22:20:29.7133333+00:00

Hi @KURAMOTOAHUJATAIRA-3118 We still have not heard back from you. Just wanted to check if the answer provided below was helpful? If it answers your query, please do click Accept Answer and Yes for the answer, as it might be beneficial to other community members reading this thread. And, if you have any further query do let us know. Thank you

Answer 1

LeelaRajeshSayana-MSFT 17,771 Moderator

Hi @KURAMOTOAHUJATAIRA-3118 Greetings! Thank you for posting the question on this forum.

Inspect the Diagnostic settings of the Azure IoT Hub and verify if Direct Methods category is enabled to be logged into the Diagnostic logs? To check this, navigate to the Diagnostic Settings under the Monitoring section of the Azure IoT Hub. If you have an existing set up that is created to log the events, Edit it to check if Direct Method is selected. Refer the below image for details.

User's image

Kindly note that the approach of your DirectMethod implementation impacts the Category of the event logs. For example, if you create the Direct method using the azure-iot-device.Client class to design Direct Method, the event generated when the method is triggered will be categorized as DirectMethods. However, if you use azure-iothub.JobClient class to create a job, the generated event will be categorized as JobsOperations.

Please refer the following resources for reference.

Let us know if you have any additional questions or need further clarification in the comments below.

If the response helped, please do click Accept Answer and Yes. Doing so would help other community members with similar issue identify the solution. I highly appreciate your contribution to the community.

KURAMOTOAHUJATAIRA-3118 250 Reputation points

2023-05-15T03:18:46.2+00:00

Hi! @LeelaRajeshSayana-MSFT

Thank you for your response.My Diagnostic settings of the Azure IoT Hub are set to collect all logs.

The current status when the query was executed on the logs is as follows.

The following is the IoT Hub query screen. It can be confirmed that the history of executing Direct Method remains on the IoT Hub. What could be the reason that it cannot be confirmed from the logs?
LeelaRajeshSayana-MSFT 17,771 Reputation points Moderator

2023-05-16T02:12:49.99+00:00

Hi @KURAMOTOAHUJATAIRA-3118 Thank you for your response. We are investigating this issue further based on your feedback. Please stay tuned on this thread for updates in the next few days.
LeelaRajeshSayana-MSFT 17,771 Reputation points Moderator

2023-05-17T23:32:11.86+00:00

Hi @KURAMOTOAHUJATAIRA-3118 Can you please confirm how you have created the direct methods on the device? I have followed this tutorial Create a device app with a direct method and created a direct method. I then called the direct method from a Service app. I noticed the method call is registered in the logs.

However, I could not find the direct methods when I queried the devices.job.

I suspect there might be a different approach you have followed to create these jobs. I appreciate it if you can share them with me for further investigation.
KURAMOTOAHUJATAIRA-3118 250 Reputation points

2023-05-18T03:57:54.85+00:00
Hi @LeelaRajeshSayana-MSFT
As you mentioned, it appears that I am handling jobs in a different way than the method described in the link you provided. I am using the Job Client Class from the SDK to schedule jobs. Here is an excerpt of the script:

const iothubJobClient = require("azure-iothub").JobClient; const jobClient = iothubJobClient.fromConnectionString( systemConfig.iothub.registryReadWriteConnectionStr); return jobClient.scheduleDeviceMethod( scheduleId, // Job ID (Schedule ID) pgFormat(queryCondition, ...replacements), // Query condition methodParams, // Method parameters scheduleDate, // Job start time 180 // Maximum execution time (seconds) );

There has been some progress. I was able to view job-related logs with the following query. It seems like I can achieve what I want, but I'm still concerned that the Category for Direct Method is not displayed. Could this be due to the difference in scheduling methods?

AzureDiagnostics | where ResourceProvider == "MICROSOFT.DEVICES" and ResourceType == "IOTHUBS" | where Category == "JobsOperations"
LeelaRajeshSayana-MSFT 17,771 Reputation points Moderator

2023-05-18T14:25:35.9066667+00:00

Hi @KURAMOTOAHUJATAIRA-3118 Thank you so much for your response and sharing your observations on this. I am glad you could catch the events by setting the category to JobsOperations. It looks like that is the category the jobs are being set to following the approach you have shared.

Please also refer this article Device management with Node.js which provides a sample direct method using ('azure-iot-device').Client class that will categorize the method created to DirectMethod.

I have updated the answer provided above to include all these observations. I appreciate it if you can Accept Answer and provide feedback on this thread to benefit other community members that might run into same issue.
KURAMOTOAHUJATAIRA-3118 250 Reputation points

2023-05-23T00:37:58.7166667+00:00

Thank you very much for your help!

Share via

Can all communication logs between devices and the cloud be obtained from IoT Hub logs, associated with each device ID?

0 additional answers

Your answer