Edit

Customize repository deployments (Public Preview)

  • Article

There are two primary ways to customize the deployment of your repository content to Microsoft Sentinel workspaces. Each method uses different files and syntax, so consider these examples to get you started.

  • Modify the GitHub workflow or DevOps pipeline to customize deployment options such as your connection's deployment trigger, deployment path, or usage of smart deployments.

  • Utilize the newly introduced configuration file to control the prioritized order of your content deployments, choose to exclude specific content files from those deployments, or map parameter files to specific content files.

Important

The Microsoft Sentinel Repositories feature is currently in PREVIEW. See the Supplemental Terms of Use for Microsoft Azure Previews for additional legal terms that apply to Azure features that are in beta, preview, or otherwise not yet released into general availability.

Prerequisites and scope

Microsoft Sentinel currently supports connections to GitHub and Azure DevOps repositories. Before connecting your Microsoft Sentinel workspace to your source control repository, make sure that you have:

  • An Owner role in the resource group that contains your Microsoft Sentinel workspace or a combination of User Access Administrator and Sentinel Contributor roles to create the connection
  • Collaborator access to your GitHub repository or Project Administrator access to your Azure DevOps repository
  • Actions enabled for GitHub and Pipelines enabled for Azure DevOps
  • Ensure custom content files you want to deploy to your workspaces are in relevant Azure Resource Manager (ARM) templates

For more information, see Validate your content.

Customize the workflow or pipeline

The default workflow only deploys content modified since the last deployment, based on commits to the repository. But you may need other customizations such as to configure different deployment triggers, or to deploy content exclusively from a specific root folder.

Select one of the following tabs depending on your connection type:

To customize your GitHub deployment workflow:

  1. In GitHub, go to your repository and find your workflow in the .github/workflows directory.

    The workflow file is the YML file starting with sentinel-deploy-xxxxx.yml. Open that file and the workflow name is shown in the first line and has the following default naming convention: Deploy Content to <workspace-name> [<deployment-id>].

    For example: name: Deploy Content to repositories-demo [xxxxx-dk5d-3s94-4829-9xvnc7391v83a]

  2. Select the pencil button at the top-right of the page to open the file for editing, and then modify the deployment as follows:

    • To modify the deployment trigger, update the on section in the code, which describes the event that triggers the workflow to run.

      By default, this configuration is set to on: push, which means that the workflow is triggered at any push to the connected branch, including both modifications to existing content and additions of new content to the repository. For example:

      on:
    push:
        branches: [ main ]
        paths:
        - `**`
        - `!.github/workflows/**` # this filter prevents other workflow changes from triggering this workflow
        - `.github/workflows/sentinel-deploy-<deployment-id>.yml`

      You may want to change these settings, for example, to schedule the workflow to run periodically, or to combine different workflow events together.

      For more information, see the GitHub documentation on configuring workflow events.

    • To disable smart deployments: The smart deployments behavior is separate from the deployment trigger discussed. Navigate to the jobs section of your workflow. Switch the smartDeployment default value from true to false. Once this change is committed, the smart deployment functionality is turned off, and all future deployments for this connection redeploy all the repository's relevant content files to the connected workspaces.

    • To modify the deployment path:

      In the default configuration shown for the on section, the wildcards (**) in the first line in the paths section indicate that the entire branch is in the path for the deployment triggers.

      This default configuration means that a deployment workflow is triggered anytime that content is pushed to any part of the branch.

      Later on in the file, the jobs section includes the following default configuration: directory: '${{ github.workspace }}'. This line indicates that the entire GitHub branch is in the path for the content deployment, without filtering for any folder paths.

      To deploy content from a specific folder path only, add it to both the paths and the directory configuration. For example, to deploy content only from a root folder named SentinelContent, update your code as follows:

      paths:
- `SentinelContent/**`
- `!.github/workflows/**` # this filter prevents other workflow changes from triggering this workflow
- `.github/workflows/sentinel-deploy-<deployment-id>.yml`

...
    directory: '${{ github.workspace }}/SentinelContent'

For more information, see the GitHub documentation on GitHub Actions and editing GitHub workflows.

Important

In both GitHub and Azure DevOps, make sure that you keep the trigger path and deployment path directories consistent.

Scale your deployments with parameter files

Rather than passing parameters as inline values in your content files, consider using a JSON file that contains the parameter values. Then map those parameter JSON files to their associated Sentinel content files to better scale your deployments across different workspaces. There are several ways to map parameter files to Sentinel files, and the repositories deployment pipeline considers them in the following order:

A diagram showing the precedence of parameter file mappings.

  1. Is there a mapping in the sentinel-deployment.config? For more information, see Customize your connection configuration.
  2. Is there a workspace-mapped parameter file? Yes it is a parameter file in the same directory as the content files that ends with .parameters-<WorkspaceID>.json
  3. Is there a default parameter file? Yes, any parameter file in the same directory as the content files that ends with .parameters.json

It is encouraged to map your parameter files through the configuration file or by specifying the workspace ID in the file name to avoid clashes in scenarios with multiple deployments.

Important

Once a parameter file match is determined based on the above mapping precedence, the pipeline will ignore any remaining mappings.

Modifying the mapped parameter file listed in the sentinel-deployment.config triggers the deployment of its paired content file. Adding or modifying a .parameters-<WorkspaceID>.json file or .parameters.json file also triggers a deployment of the paired content files along with the newly modified parameters, unless a higher precedence parameter mappings is in place. Other content files aren't deployed as long as the smart deployments feature is still enabled in the workflow/pipeline definition file.

Customize your connection configuration

The deployment script for repositories supports the usage of a deployment configuration file for each repository branch as of July 2022. The configuration JSON file helps you map parameter files to relevant content files, prioritize specific content in deployments, and exclude specific content from deployments.

  1. Create the file sentinel-deployment.config at the root of your repository. Adding, deleting, or modifying this configuration file will cause a full deployment of all the content in the repository according to the updated configuration.

    Screenshot of a repository root directory. The RepositoriesSampleContent is shown with the location of the sentinel-deployment.config file.

  2. Include JSON structured content in three optional sections, "prioritizedcontentfiles":, "excludecontentfiles":, and "parameterfilemappings":. If no sections are included or the .config file is omitted, the deployment process still runs. Invalid or unrecognized sections are ignored.

Here's an example of the entire contents of a valid sentinel-deployment.config file. This sample can also be found at the Sentinel CICD repositories sample.

{
  "prioritizedcontentfiles": [
    "parsers/Sample/ASimAuthenticationAWSCloudTrail.json",
    "workbooks/sample/TrendMicroDeepSecurityAttackActivity_ARM.json",
    "Playbooks/PaloAlto-PAN-OS/PaloAltoCustomConnector/azuredeploy.json"
  ], 
  "excludecontentfiles": [
     "Detections/Sample/PaloAlto-PortScanning.json",
     "parameters"
  ],
  "parameterfilemappings": {
    "879001c8-2181-4374-be7d-72e5dc69bd2b": {
      "Playbooks/PaloAlto-PAN-OS/Playbooks/PaloAlto-PAN-OS-BlockIP/azuredeploy.json": "parameters/samples/parameter-file-1.json"
    },
    "9af71571-7181-4cef-992e-ef3f61506b4e": {
      "Playbooks/Enrich-SentinelIncident-GreyNoiseCommunity-IP/azuredeploy.json": "path/to/any-parameter-file.json"
    }
  },
  "DummySection": "This shouldn't impact deployment"
}

Note

Don't use the backslash "\" character in any of the content paths. Use the forward slash "/" instead.

  • To prioritize content files:

    As the amount of content in your repository grows, deployment times may increase. Add time sensitive content to this section to prioritize its deployment when a trigger occurs.

    Add full path names to the "prioritizedcontentfiles": section. Wildcard matching is not supported at this time.

  • To exclude content files, modify the "excludecontentfiles": section with full path names of individual .json content files.

  • To map parameters:

    The deployment script accepts three methods of mapping parameters as described in Scale your deployments with parameter files. Mapping parameters through the sentinel-deployment.config takes the highest precedence and guarantees that a given parameter file is mapped to its associated content files. Simply modify the "parameterfilemappings": section with your target connection's workspace ID and full path names of individual .json files.

Next steps

A sample repository is available demonstrating the deployment config file and all three parameter mapping methods. For more information, see Sentinel CICD repositories sample.

Consider these resources for more information about ARM templates: