Hello, I would like to bring your attention to an issue we are currently experiencing: We have an Azure Data Factory (ADF) setup and are trying to configure logging for it. A Log Analytics workspace has been created and attached to ADF. We set up diagnostic settings and enabled logging for pipeline activity runs , pipeline runs , and trigger runs . The destination for the logs was set to the Log Analytics workspace , with the Resource Specific table selected. However, it has been almost 24 hours since we enabled the logging, but we still do not see any logs within the Log Analytics workspace. Any guidance on how to troubleshoot or verify the configuration would be greatly appreciated.

@Sai Kumar Thadepalli I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that " The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer . Ask: I would like to bring your attention to an issue we are currently experiencing: We have an Azure Data Factory (ADF) setup and are trying to configure logging for it. A Log Analytics workspace has been created and attached to ADF. We set up diagnostic settings and enabled logging for pipeline activity runs , pipeline runs , and trigger runs . The destination for the logs was set to the Log Analytics workspace , with the Resource Specific table selected. However, it has been almost 24 hours since we enabled the logging, but we still do not see any logs within the Log Analytics workspace. Any guidance on how to troubleshoot or verify the configuration would be greatly appreciated. Solution: I checked with the team again, but that was solved while it got recreated with terraform, it sounds more like a permissions issue to me, though I haven't received much updates on that what solved the issue. If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information. If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue. Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.

ADF is unable to place logs inside Log analytics workspace.

Sai Kumar Thadepalli 20

Hello,

I would like to bring your attention to an issue we are currently experiencing:

We have an Azure Data Factory (ADF) setup and are trying to configure logging for it.

A Log Analytics workspace has been created and attached to ADF.
We set up diagnostic settings and enabled logging for pipeline activity runs, pipeline runs, and trigger runs.
The destination for the logs was set to the Log Analytics workspace, with the Resource Specific table selected.

However, it has been almost 24 hours since we enabled the logging, but we still do not see any logs within the Log Analytics workspace.

Any guidance on how to troubleshoot or verify the configuration would be greatly appreciated.

phemanth 13,060 Reputation points Microsoft Vendor

2024-12-06T18:54:12.44+00:00
Hi@Sai Kumar Thadepalli

Thanks for reaching out to Microsoft Q&A.

Here are a few steps you can take to troubleshoot and verify your configuration:

Ensure that the diagnostic settings are correctly configured to send logs to the Log Analytics workspace. Double-check that the correct Log Analytics workspace is selected and that the necessary log categories (pipeline activity runs, pipeline runs, and trigger runs) are enabled.

Ensure that the user configuring the diagnostic settings has the necessary permissions to write to the Log Analytics workspace. The user should have at least the Contributor role on both the Azure Data Factory and the Log Analytics workspace.

Sometimes, there can be a delay in log ingestion. Although 24 hours is typically sufficient, it might be worth waiting a bit longer to see if the logs appear.

Use the Log Analytics query interface to manually search for logs. Sometimes, logs might be present but not immediately visible due to query settings. You can use queries like:
ADFPipelineRun | where TimeGenerated > ago(1d)

Occasionally, issues can be region-specific. Ensure that there are no ongoing outages or issues in the region where your ADF and Log Analytics workspace are hosted.

I hope these steps help you resolve the issue. Please Let me know if issue persists.
Sai Kumar Thadepalli 20 Reputation points

2024-12-07T06:38:50.0766667+00:00
Hi @phemanth,

Thanks for responding,

To confirm, The first & all other points are verified.

We will check the permissions of who created the Log analytics workspace. I believe he is having the contributor access because most of the deployments will be dealt by that team.

I will look at verifying the contributor access at ADF side as well.

Could you please confirm me if there are any other permissions required at log analytics workspace level for service to service connection? like Managed identity permissions.
phemanth 13,060 Reputation points Microsoft Vendor

2024-12-10T03:32:55.0166667+00:00
@Sai Kumar Thadepalli

It’s great to hear that you’ve verified the initial points. Regarding additional permissions, here are some key permissions and roles that might be necessary for a service-to-service connection, such as using a Managed Identity:

Log Analytics Workspace Permissions:

Reader Role: The Managed Identity should have the Log Analytics Reader role to view logs.

Contributor Role: The Log Analytics Contributor role is needed to read and edit workspace settings.

Specific Permissions: Ensure the Managed Identity has the following permissions:

Microsoft.OperationalInsights/workspaces/read

Microsoft.OperationalInsights/workspaces/query/read

Microsoft.OperationalInsights/workspaces/sharedKeys/action1.

Azure Data Factory Permissions:

The Managed Identity should have the necessary permissions to access and write logs to the Log Analytics workspace. This typically includes the Contributor role on both the ADF and the Log Analytics workspace.

Resource-Specific Permissions:

If your workspace is set to use resource-context mode, ensure that the Managed Identity has read permissions on the specific resources sending logs to the workspace.

By ensuring these permissions are in place, you should be able to establish a proper service-to-service connection and facilitate logging from ADF to your Log Analytics workspace.
phemanth 13,060 Reputation points Microsoft Vendor

2024-12-12T16:52:55.0333333+00:00

@Sai Kumar Thadepalli We haven’t heard from you on the last response and was just checking back to see if you have a resolution yet. In case if you have any resolution please do share that same with the community as it can be helpful to others. Otherwise, will respond with more details and we will try to help.
Sai Kumar Thadepalli 20 Reputation points

2024-12-13T06:25:10.8666667+00:00

Hey @phemanth,

I checked with the team again, but that was solved while it got recreated with terraform, it sounds more like a permissions issue to me, though I haven't received much updates on that what solved the issue.
phemanth 13,060 Reputation points Microsoft Vendor

2024-12-13T17:03:15.0066667+00:00

Glad to know your issue has been resolved. Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others "I'll repost your solution in case you'd like to accept the answer.

Accepted answer

phemanth 13,060 Reputation points Microsoft Vendor

2024-12-13T17:03:23.81+00:00
@Sai Kumar Thadepalli

I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others ", I'll repost your solution in case you'd like to accept the answer .

Ask:

I would like to bring your attention to an issue we are currently experiencing:

We have an Azure Data Factory (ADF) setup and are trying to configure logging for it.

A Log Analytics workspace has been created and attached to ADF.

We set up diagnostic settings and enabled logging for pipeline activity runs, pipeline runs, and trigger runs.

The destination for the logs was set to the Log Analytics workspace, with the Resource Specific table selected.

However, it has been almost 24 hours since we enabled the logging, but we still do not see any logs within the Log Analytics workspace.

Any guidance on how to troubleshoot or verify the configuration would be greatly appreciated.

Solution: I checked with the team again, but that was solved while it got recreated with terraform, it sounds more like a permissions issue to me, though I haven't received much updates on that what solved the issue.

If I missed anything please let me know and I'd be happy to add it to my answer, or feel free to comment below with any additional information.

If you have any other questions, please let me know. Thank you again for your time and patience throughout this issue.

Please don’t forget to Accept Answer and Yes for "was this answer helpful" wherever the information provided helps you, this can be beneficial to other community members.
Please sign in to rate this answer.

1 person found this answer helpful.
phemanth 13,060 Reputation points Microsoft Vendor

2024-12-16T16:51:15.13+00:00

@Sai Kumar Thadepalli Following up to see if the above answer was helpful. If this answers your query, do click Accept Answer and Yes for was this answer helpful. And, if you have any further query do let us know.
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

ADF is unable to place logs inside Log analytics workspace.

0 additional answers

Your answer