Explore advanced threat hunting in Microsoft Defender XDR
Advanced hunting in Microsoft Defender XDR is a query-based threat-hunting tool that lets you explore up to 30 days of raw data. You can proactively inspect events in your network to locate threat indicators and entities. The flexible access to data enables unconstrained hunting for both known and potential threats.
Advanced hunting enables you to proactively hunt for threats across:
- Devices managed by Microsoft Defender for Endpoint.
- Emails processed by Microsoft 365.
- Cloud app activities, authentication events, and domain controller activities tracked by Microsoft Cloud App Security and Microsoft Defender for Identity.
With this level of visibility, you can quickly hunt for threats that traverse sections of your network. These threats include sophisticated intrusions that:
- Arrive on email or the web.
- Elevate local privileges.
- Acquire privileged domain credentials.
- Move laterally across your devices.
You can use the same threat-hunting queries to build custom detection rules. These rules run automatically to check for and then respond to various events and system states. These events include suspected breach activity, misconfigured machines, and other findings.
Get started with advanced hunting in Microsoft Defender XDR
Microsoft recommends that organizations complete several steps to quickly get started with advanced hunting in Microsoft Defender XDR:
- Learn the language. Advanced hunting is based on Kusto query language, supporting the same syntax and operators. Start learning the query language by running your first query.
- Learn how to use the query results. Learn about charts and various ways you can view or export your results. Explore how you can quickly tweak queries, drill down to get richer information, and take response actions.
- Understand the schema. Get a good, high-level understanding of the tables in the schema and their columns. Learn where to look for data when constructing your queries.
- Get expert tips and examples. Train for free with guides from Microsoft experts. Explore collections of predefined queries covering different threat hunting scenarios.
- Optimize queries and handle errors. Understand how to create efficient and error-free queries.
- Create custom detection rules. Understand how you can use advanced hunting queries to trigger alerts and take response actions automatically.
You can categorize advanced hunting data into two distinct types, each consolidated differently.
- Event or activity data. Populates tables about alerts, security events, system events, and routine assessments. Advanced hunting receives this data almost immediately after the sensors that collect them successfully transmit them to the corresponding cloud services. For example, you can query event data from healthy sensors on workstations or domain controllers almost immediately after they're available on Microsoft Defender for Endpoint and Microsoft Defender for Identity.
- Entity data. Populates tables with information about users and devices. This data comes from both relatively static data sources and dynamic sources, such as Active Directory entries and event logs. To provide fresh data, the system updates tables with any new information every 15 minutes. It also adds rows even if it doesn't fully populate them. Every 24 hours, the system combines the data to insert a record that contains the latest, most comprehensive data set about each entity.
Introduction to the Kusto query language used by advanced hunting
Advanced hunting is based on the Kusto query language. A Kusto query is a read-only request to process data and return results. You write the request in plain text, using a data-flow model designed to make the syntax easy to read, author, and automate. The query uses schema entities organized in a hierarchy similar to SQL Server's schema design: databases, tables, and columns.
Important
The purpose of this section on the Kusto query language is to introduce you to this feature. This section by no means provides an in-depth analysis of the language specifics. Since advanced hunting in Microsoft Defender XDR relies on Kusto queries, it's important that you have a basic understanding of the general concepts behind the language. For more information, see Tutorial: Use Kusto queries in Azure Data Explorer and Azure Monitor.
A Kusto query consists of a sequence of query statements, delimited by a semicolon. At least one statement must be a tabular expression statement, which is a statement that produces data arranged in a table-like mesh of columns and rows. The query's tabular expression statements produce the results of the query.
The syntax of the tabular expression statement has tabular data flow from one tabular query operator to another. The flow starts with the data source (for example, a table in a database, or an operator that produces data) and then flows through a set of data transformation operators. The statement binds the operators together with the pipe (|) delimiter.
For example, consider the following query statement that queries the number of storms that occurred in the state of Florida over a give time period. The following Kusto query has a single statement, which is a tabular expression statement. The statement starts with a reference to a table called StormEvents (the database that host this table is implicit here, and part of the connection information). The system then filters the data (rows) for that table by the value of the StartTime column, and then filters again using the value of the State column. The query then returns the count of "surviving" rows.
A Kusto query statement uses the following operators:
- Where. Filter a table to the subset of rows that satisfy a predicate.
- Summarize. Produce a table that aggregates the content of the input table.
- Join. Merge the rows of two tables to form a new table by matching values of the specified column(s) from each table.
- Count. Return the number of records in the input record set.
- Top. Return the first N records sorted by the specified columns.
- Limit. Return up to the specified number of rows.
- Project. Select the columns to include, rename or drop, and insert new computed columns.
- Extend. Create calculated columns and append them to the result set.
- Makeset(). Return a dynamic (JSON) array of the set of distinct values that takes in the group.
- Find. Find rows that match a predicate across a set of tables.
The following screenshot shows an advanced hunting query using the Kusto query language. This query uses the following filters:
- Time filter to review only records from the previous seven days.
- Filter on the FileName to contain only instances of Powershell.EXE.
- Filter on the ProcessCommandLine.
- Project only the columns you're interested in exploring and limit the results to 100.
To ensure query performance, organizations should implement the following best practices when writing Kusto query statements:
- Apply filters first. Kusto highly optimizes the use of time filters.
- Use the "has" keyword rather than "contains" when looking for full tokens.
- Use looking in specific column rather than using full text search across all columns.
- When joining between two tables, choose the table with fewer rows to be the first one (left-most).
- When joining between two tables, project only needed columns from both sides of the join.
Here's an example of a Kusto advanced hunting query. This query searches for file creation events where the created files had suspicious file extensions:
Knowledge check
Choose the best response for the following question. Then select “Check your answers.”