Connect Microsoft Sentinel to Azure, Windows, Microsoft, and Amazon services

Note

For information about feature availability in US Government clouds, see the Microsoft Sentinel tables in Cloud feature availability for US Government customers.

Microsoft Sentinel uses the Azure foundation to provide built-in, service-to-service support for data ingestion from many Azure and Microsoft 365 services, Amazon Web Services, and various Windows Server services. There are a few different methods through which these connections are made, and this article describes how to make these connections.

This article describes the collection of Windows Security Events. For Windows DNS events, learn about the Windows DNS Events via AMA connector (Preview).

Types of connections

This article discusses the following types of connectors:

  • API-based connections
  • Diagnostic settings connections, some of which are managed by Azure Policy
  • Log Analytics agent-based connections

This article presents information that is common to groups of connectors. See the accompanying data connector reference page for information that is unique to each connector, such as licensing prerequisites and Log Analytics tables for data storage.

The following integrations are both more unique and more popular, and are treated individually, with their own articles:

API-based connections

Prerequisites

  • You must have read and write permissions on the Log Analytics workspace.
  • You must have the Global administrator or Security administrator role on your Microsoft Sentinel workspace's tenant.

Instructions

  1. From the Microsoft Sentinel navigation menu, select Data connectors.

  2. Select your service from the data connectors gallery, and then select Open Connector Page on the preview pane.

  3. Select Connect to start streaming events and/or alerts from your service into Microsoft Sentinel.

  4. If on the connector page there is a section titled Create incidents - recommended!, select Enable if you want to automatically create incidents from alerts.

You can find and query the data for each service using the table names that appear in the section for the service's connector in the Data connectors reference page.

Diagnostic settings-based connections

The configuration of some connectors of this type is managed by Azure Policy. Select the Azure Policy tab below for instructions. For the other connectors of this type, select the Standalone tab.

Prerequisites

To ingest data into Microsoft Sentinel:

  • You must have read and write permissions on the Microsoft Sentinel workspace.

Instructions

  1. From the Microsoft Sentinel navigation menu, select Data connectors.

  2. Select your resource type from the data connectors gallery, and then select Open Connector Page on the preview pane.

  3. In the Configuration section of the connector page, select the link to open the resource configuration page.

    If presented with a list of resources of the desired type, select the link for a resource whose logs you want to ingest.

  4. From the resource navigation menu, select Diagnostic settings.

  5. Select + Add diagnostic setting at the bottom of the list.

  6. In the Diagnostics settings screen, enter a name in the Diagnostic settings name field.

    Mark the Send to Log Analytics check box. Two new fields will be displayed below it. Choose the relevant Subscription and Log Analytics Workspace (where Microsoft Sentinel resides).

  7. Mark the check boxes of the types of logs and metrics you want to collect. See our recommended choices for each resource type in the section for the resource's connector in the Data connectors reference page.

  8. Select Save at the top of the screen.

For more information, see also Create diagnostic settings to send Azure Monitor platform logs and metrics to different destinations in the Azure Monitor documentation.

Note

With this type of data connector, the connectivity status indicators (a color stripe in the data connectors gallery and connection icons next to the data type names) will show as connected (green) only if data has been ingested at some point in the past 14 days. Once 14 days have passed with no data ingestion, the connector will show as being disconnected. The moment more data comes through, the connected status will return.

You can find and query the data for each resource type using the table name that appears in the section for the resource's connector in the Data connectors reference page. For more information, see Create diagnostic settings to send Azure Monitor platform logs and metrics to different destinations in the Azure Monitor documentation.

Windows agent-based connections

Note

The Windows DNS Events via AMA connector (Preview) also uses the Azure Monitor Agent. This connector streams and filter events from Windows Domain Name System (DNS) server logs.

Important

Some connectors based on the Azure Monitor Agent (AMA) are currently in PREVIEW. See the Supplemental Terms of Use for Microsoft Azure Previews for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

The Azure Monitor Agent is currently supported only for Windows Security Events and Windows Forwarded Events.

The Azure Monitor agent uses Data collection rules (DCRs) to define the data to collect from each agent. Data collection rules offer you two distinct advantages:

  • Manage collection settings at scale while still allowing unique, scoped configurations for subsets of machines. They are independent of the workspace and independent of the virtual machine, which means they can be defined once and reused across machines and environments. See Configure data collection for the Azure Monitor agent.

  • Build custom filters to choose the exact events you want to ingest. The Azure Monitor Agent uses these rules to filter the data at the source and ingest only the events you want, while leaving everything else behind. This can save you a lot of money in data ingestion costs!

See below how to create data collection rules.

Prerequisites

  • You must have read and write permissions on the Microsoft Sentinel workspace.

  • To collect events from any system that is not an Azure virtual machine, the system must have Azure Arc installed and enabled before you enable the Azure Monitor Agent-based connector.

    This includes:

    • Windows servers installed on physical machines
    • Windows servers installed on on-premises virtual machines
    • Windows servers installed on virtual machines in non-Azure clouds

Instructions

  1. From the Microsoft Sentinel navigation menu, select Data connectors. Select your connector from the list, and then select Open connector page on the details pane. Then follow the on-screen instructions under the Instructions tab, as described through the rest of this section.

  2. Verify that you have the appropriate permissions as described under the Prerequisites section on the connector page.

  3. Under Configuration, select +Add data collection rule. The Create data collection rule wizard will open to the right.

  4. Under Basics, enter a Rule name and specify a Subscription and Resource group where the data collection rule (DCR) will be created. This does not have to be the same resource group or subscription the monitored machines and their associations are in, as long as they are in the same tenant.

  5. In the Resources tab, select +Add resource(s) to add machines to which the Data Collection Rule will apply. The Select a scope dialog will open, and you will see a list of available subscriptions. Expand a subscription to see its resource groups, and expand a resource group to see the available machines. You will see Azure virtual machines and Azure Arc-enabled servers in the list. You can mark the check boxes of subscriptions or resource groups to select all the machines they contain, or you can select individual machines. Select Apply when you've chosen all your machines. At the end of this process, the Azure Monitor Agent will be installed on any selected machines that don't already have it installed.

  6. On the Collect tab, choose the events you would like to collect: select All events or Custom to specify other logs or to filter events using XPath queries (see note below). Enter expressions in the box that evaluate to specific XML criteria for events to collect, then select Add. You can enter up to 20 expressions in a single box, and up to 100 boxes in a rule.

    Learn more about data collection rules from the Azure Monitor documentation.

    Note

    • The Windows Security Events connector offers two other pre-built event sets you can choose to collect: Common and Minimal.

    • The Azure Monitor agent supports XPath queries for XPath version 1.0 only.

  7. When you've added all the filter expressions you want, select Next: Review + create.

  8. When you see the "Validation passed" message, select Create.

You'll see all your data collection rules (including those created through the API) under Configuration on the connector page. From there you can edit or delete existing rules.

Tip

Use the PowerShell cmdlet Get-WinEvent with the -FilterXPath parameter to test the validity of an XPath query. The following script shows an example:

$XPath = '*[System[EventID=1035]]'
Get-WinEvent -LogName 'Application' -FilterXPath $XPath
  • If events are returned, the query is valid.
  • If you receive the message "No events were found that match the specified selection criteria," the query may be valid, but there are no matching events on the local machine.
  • If you receive the message "The specified query is invalid," the query syntax is invalid.

Create data collection rules using the API

You can also create data collection rules using the API (see schema), which can make life easier if you're creating many rules (if you're an MSSP, for example). Here's an example (for the Windows Security Events via AMA connector) that you can use as a template for creating a rule:

Request URL and header

PUT https://management.azure.com/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.Insights/dataCollectionRules/myCollectionRule?api-version=2019-11-01-preview

Request body

{
    "location": "eastus",
    "properties": {
        "dataSources": {
            "windowsEventLogs": [
                {
                    "streams": [
                        "Microsoft-SecurityEvent"
                    ],
                    "xPathQueries": [
                        "Security!*[System[(EventID=) or (EventID=4688) or (EventID=4663) or (EventID=4624) or (EventID=4657) or (EventID=4100) or (EventID=4104) or (EventID=5140) or (EventID=5145) or (EventID=5156)]]"
                    ],
                    "name": "eventLogsDataSource"
                }
            ]
        },
        "destinations": {
            "logAnalytics": [
                {
                    "workspaceResourceId": "/subscriptions/703362b3-f278-4e4b-9179-c76eaf41ffc2/resourceGroups/myResourceGroup/providers/Microsoft.OperationalInsights/workspaces/centralTeamWorkspace",
                    "name": "centralWorkspace"
                }
            ]
        },
        "dataFlows": [
            {
                "streams": [
                    "Microsoft-SecurityEvent"
                ],
                "destinations": [
                    "centralWorkspace"
                ]
            }
        ]
    }
}

See this complete description of data collection rules from the Azure Monitor documentation.

Next steps

In this document, you learned how to connect Azure, Microsoft, and Windows services, as well as Amazon Web Services, to Microsoft Sentinel.