Prepare for attacks with Attack simulation training

  • 4 minutes

Organizations that have Microsoft 365 E5 or Microsoft Defender for Office 365 Plan 2 can use Attack simulation training to run realistic attack scenarios. This feature appears in the Microsoft Defender portal.

Note

Microsoft Defender for Office 365 Plan 2 also includes Threat Investigation and Response capabilities.

Simulated attacks can help an organization identify its vulnerable users. With this information, organizations can hopefully change the behavior of these individuals before a real attack impacts the organization's bottom line.

Attack simulation training in Microsoft Defender for Office 365 lets an organization run benign cyberattack simulations to test its security policies and practices. It also enables organizations to train employees to increase their awareness and reduce their susceptibility to attacks.

To access Attack simulation training, you should navigate to the Microsoft Defender portal and select Email and collaboration > Attack simulation training.

An organization must meet the following prerequisites to run Attack simulation training:

  • The organization must have either a Microsoft 365 E5 subscription, or Microsoft Defender for Office 365 Plan 2.
  • The user running the attack simulation training must have one of the following Microsoft 365 roles:
    • Global Administrator
    • Security Administrator
    • Either of the following roles designed specifically for Attack simulation training:
      • Attack Simulator Administrators. Create and manage all aspects of attack simulation campaigns.
      • Attack Simulator Payload Authors. Create attack payloads that an admin can start later.
  • Exchange Online hosts the organization's email. Attack simulation training isn't available for on-premises email servers.
  • The organization must store its attack simulation data and training-related data with other customer data for its Microsoft 365 services. For more information, see Where Microsoft 365 stores your customer data.

Understanding user behavior

Attack simulation training provides insight based on:

  • The data generated by the simulations.
  • The trainings each employee completed.

This knowledge helps administrators stay informed on the threat readiness of each employee within the organization. Attack simulation training can also recommend the next steps an organization should perform to strengthen itself from attacks.

To review the data generated by the simulations, an administrator must select the Overview tab from the Attack simulation training page. From this tab, an administrator can view the behavior impact on compromise rate card. This card shows how employees dealt with the simulations the administrator ran in contrast to the predicted compromise rate. Administrators can use this information to track progress in employees' threat readiness. They can do so by running multiple simulations against the same groups of employees.

The graph depicts:

  • Actual compromise rate. Reflects the percentage of employees that fell for the simulation.
  • Predicted compromise rate. Reflects the average compromise rate for simulations using the same type of payload across other Microsoft 365 tenants that use Attack simulation training.

Screenshot of the attack simulation training graph.

Running Attack simulation training

Attack simulation training consists of the following steps.

  1. Select a social engineering (simulation) technique.
  2. Select a payload.
  3. Audience targeting.
  4. Assign training.
  5. Launch details and review.

The following sections describe each of these steps in greater detail.

Step 1 - Select a social engineering (simulation) technique

Phishing is a generic term for email attacks that try to steal sensitive information in messages that appear to be from legitimate or trusted senders. Phishing is part of a subset of techniques Microsoft classifies as social engineering.

In Attack simulation training, you can select from the following social engineering (simulation) techniques:

  • Credential harvest. An attacker sends the recipient a message that contains a URL. When the recipient selects the URL, the system directs the user to a website that typically shows a username and password sign-in box. The attacker designs the destination page to represent a well-known website. The goal of the phishing attack is to have the user believe they accessed a real site.
  • Malware attachment. An attacker sends the recipient a message that contains an attachment. When the recipient opens the attachment, arbitrary code (for example, a macro) automatically runs on the user's device. This code helps the attacker install more code or further entrench themselves.
  • Link in attachment. This simulation is a hybrid of a credential harvest. An attacker sends the recipient a message that contains a URL within an attachment. When the recipient opens the attachment and selects the URL, they get routed to a website that typically shows a sign-in dialog box. This site asks the user for their username and password. By displaying what appears to be a well-known website, the goal of the phishing attack is to have the user actually believe they accessed a real site.
  • Link to malware. An attacker sends the recipient a message that contains a link to an attachment on a well-known file sharing site (for example, SharePoint Online or Dropbox). When the recipient selects the URL, the attachment opens, and then arbitrary code, such as a macro, runs on the user's device. This code helps the attacker install more code or further entrench themselves.
  • Drive-by-url. An attacker sends the recipient a message that contains a URL. When the recipient selects the URL, the system directs the user to a website that tries to run background code. This code either gathers information about the recipient or deploys arbitrary code on their device. Typically, the destination website is either a well-known website the attacker compromised, or a clone of a well-known website. Familiarity with the website helps convince the user the link is safe to select. Some people refer to drive-by-url attacks as watering hole attacks.

Step 2 - Select a payload

Once you select the type of simulation (social engineering technique) that you want to run, select a payload from the pre-existing payload catalog.

The purpose of payload data points is to provide insights into the effectiveness of an organization's security controls against simulated attacks. These data points can help organizations identify vulnerabilities in their security posture and take steps to remediate them before real attackers can exploit them.

Organizations' security teams collect and analyze data points found in payloads. They do so to gain a better understanding of how attackers might target their organization's infrastructure and data. They use this knowledge to improve their security defenses, such as by implementing:

  • Stronger access controls
  • More robust threat detection systems
  • More effective incident response processes

Security teams also use data points to track and measure the success of an organization's security awareness training program. For example, let's assume that simulation data shows a high percentage of users fell for a phishing attack. This result could indicate the users need extra training on how to recognize and avoid such attacks.

The payload catalog has many data points to help you choose a payload, including:

  • Click rate. Counts how many people selected this payload. The click rate data point is a measure of the payload's ability to trick or deceive users into taking a specific action. For example, clicking on a link or opening an attachment. The platform analyzes various factors to determine how likely users select this payload. For example, the language used in the payload, the visual design of the message, and the perceived urgency of the request.
  • Predicted compromise rate. The platform uses machine learning and other advanced algorithms to predict the likelihood that a particular payload results in a successful compromise. In doing so, the platform analyzes various factors. For example, the complexity of the payload, the likelihood of user interaction, and the effectiveness of security controls. The result of this analysis is the predicted compromise rate, which is an estimate of the percentage of users an attack might compromise. The predicted compromise rate is just an estimate. The actual rate of compromise might vary depending on various other factors. For example, awareness and training, the effectiveness of security controls, and the specific characteristics of the attack itself. As such, it's important that organizations use predicted compromise rates as a guide, rather than as a definitive measure of risk.
  • Simulations launched. The security team responsible for running simulations counts the number of times other simulations used this payload.
  • Complexity. The training platform calculates the complexity data point found in payloads. It uses algorithms and metrics to analyze the payload and assign a complexity score based on various factors. For example, the number of variables and functions used, the level of nesting, and the overall structure of the payload.
  • Source. The source indicates whether the organization's security team created the payload on its tenant, or the payload is part of Microsoft's pre-existing payload catalog (global).

Step 3 - Audience targeting

Now it's time to select this simulation's audience. You can choose to include all your users or only specific users and groups. If you choose to include only specific users and groups, you can either:

  • Add users. You can search through your tenant for specific users. You can also use advanced search and filtering capabilities. For example, you can select users who you didn't target in a simulation in the last three months.
  • Import from CSV. Enables you to import a predefined set of users for this simulation.

Step 4 - Assign training

Microsoft recommends that you assign training for each simulation. Why? Because employees who go through training are less susceptible to similar attacks. You can either choose to have training assigned for your organization, or you can select training courses and modules yourself.

Select the training due date to ensure employees finish their training to meet the organization's goals.

Step 5 - Launch details and review

Once an organization configures all the attack simulation parameters, you can launch the simulation immediately or schedule it for a later date. When launching a simulation, you must choose when to end it. The simulation stops capturing interactions when it's past the selected time.

If you set the region aware timezone delivery option when launching a simulation, simulated attack messages get delivered to your employees during their working hours based on their region.

Once the simulation finishes, select Next and review the details of your simulation. Select the Edit option on any of the parts to go back and change any details that need to change. When you feel satisfied with the results, select Submit.