Synapse spark application logs to Az Log Analytics - managed identity

Question

I want to send spark application logs to log analytics workspace and I am following the document here: https://learn.microsoft.com/en-us/azure/synapse-analytics/spark/apache-spark-azure-log-analytics#write-custom-application-logs Is there a way to achieve this with managed identity and RBAC roles? Like if I give my synapse system assigned identity contributor role to log analytics workspace, can this be done? If yes, what all spark configurations need to be set in this case?

Answer 1

Hi user_unknown,

Thanks for reaching out to Microsoft Q&A.

You can assign the system-assigned identity contributor role to your log analytics workspace. For the spark conf settings that you would need, please see the link below which explains the step-by-step instructions including the configuration settings in detail.

https://github.com/mspnp/spark-monitoring/blob/main/README.md

https://dustinvannoy.com/2022/05/12/monitor-synapse-spark-with-log-analytics/

Please 'Upvote'(Thumbs-up) and 'Accept' as an answer if the reply was helpful. This will benefit other community members who face the same issue.