Share via

How can Sentinel be used to collect K8S events on-prem connected to Azure via Azure Arc?

Jie Yin 105 Reputation points
2023-10-19T09:02:19.9133333+00:00

I have already connect the on-prem K8S to Azure via Azure Arc, and enable the Azure monitor, how to collect the K8S data using Azure Sentinel?

Microsoft Security | Microsoft Sentinel
0 comments
Accepted answer
  1. Givary-MSFT 35,626 Reputation points Microsoft Employee Moderator
    2023-11-06T07:21:22.39+00:00

    @Jie Yin I'm glad that you were able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others Opens in new window or tab", I'll repost your solution in case you'd like to "Accept Opens in new window or tab" the answer.

    Issue: connect the on-prem K8S to Azure via Azure Arc, and enable the Azure monitor, how to collect the K8S data using Azure Sentinel?

    Resolved by @Jie Yin found a open-source tools called logstash, this tools can be received the audit log from Kubernetes using Webhook, then send to Sentinel using the custom data connector of Sentinel(Use Logstash to stream logs with HTTP Data Collection API (legacy) | Microsoft Learn) .

    If you have any other questions or are still running into more issues, please let me know.
    Thank you again for your time and patience throughout this issue.

    Please remember to "Accept Answer" if any answer/reply helped, so that others in the community facing similar issues can easily find the solution.

4 additional answers

Sort by: Most helpful
Most helpful Newest Oldest
  1. Clive Watson 7,866 Reputation points MVP Volunteer Moderator
    2023-10-19T10:31:18.92+00:00

    Microsoft Sentinel can share the same Log Analytics Workspace, so if Sentinel is using the one you have sent the K8S data too, it will be visible to Sentinel - look in the Logs blade of Sentinel.
    You will then need to Hunt, or create/enable Rules to generate Incidents in Sentinel from the data.

    There are two hunting queries for AKS you maybe able to adapt: https://github.com/Azure/Azure-Sentinel/tree/274813089379496eea5c21a5300f3b8be09304a0/Solutions/Azure%20kubernetes%20Service

  2. JamesTran-MSFT 36,911 Reputation points Microsoft Employee Moderator
    2023-10-25T20:23:27.0333333+00:00

    @Jie Yin

    Thank you for your post and I apologize for the delayed response!

    I'm not familiar with Kubernetes, but to help troubleshoot your issue - since you've already connected your on-prem Kubernetes (K8S) to Azure via Azure Arc and have enabled Azure monitor to collect the K8S data. You should be able to define a data collection rule(s) and associate your resource to that rule, along with selecting the appropriate Log Analytics workspace that your Sentinel instance is using. For more info - Install the Azure Monitor Agent and configure data collection.

    If you aren't able to leverage data collection rules you can also look into leveraging the Azure Monitor Container Insights for Azure Arc-enabled Kubernetes clusters feature.

    If you're still having issues and would like to work closer with our support team on this, please let me know. I'd be happy to enable your Azure Subscription ID, for a one-time free technical support request so you can get this issue resolved.

    Additional Links:

    I hope this helps!

    If you have any other questions, please let me know. Thank you for your time and patience throughout this issue.

    If the information helped address your question, please Accept the answer. This will help us and also improve searchability for others in the community who might be researching similar information.

  3. Jie Yin 105 Reputation points
    2023-10-26T04:49:34.69+00:00

    Update：

    I was able to set up a Workbook in Sentinel to display the log information from the LA Query, as shown in the kubeEvents table below, but I didn't see the source logs for those query statements in the AKS Security Workbook that came with the type AKS connector in the collected data. Is it because Azure Arc Monitor can't collect this data?

    User's image

    AKS Data connector workbook:

    User's image

    0 comments
  4. Jie Yin 105 Reputation points
    2023-10-30T10:10:29.1066667+00:00

    After in-depth research, I have roughly figured out the process of the problem.

    In this case, first of all, the Kubernetes security and Audit logs needed by Sentinel (like the diagnostic logs set up in AKS) do not exist in the container insight (in AKS, it belongs to the resource Log), but need to be the Audit Log formed by enabling Kubernetes audit function. This Log can be stored in a local file on the PV, or transferred to the outside through APIs, Webhooks, and then can be stored in the LA. After that, it can be analyzed using a Notebook on the Sentinel associated with the LA.

    So the core of the problem is not whether Azure Arc-enabled Kubernetes is available or not, but rather that audit logs need to be enabled and somehow (either by developing an API or an application that handles files) transferred to LA.

    Thank you again for your help.

    Additional Links:

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.