GCP data connector reference for the Codeless Connector Platform

Article
10/01/2024

To create a Google Cloud Platform (GCP) data connector with the Codeless Connector Platform (CCP), use this reference as a supplement to the Microsoft Sentinel REST API for Data Connectors docs.

Each dataConnector represents a specific connection of a Microsoft Sentinel data connector. One data connector might have multiple connections, which fetch data from different endpoints. The JSON configuration built using this reference document is used to complete the deployment template for the CCP data connector.

For more information, see Create a codeless connector for Microsoft Sentinel.

Build the GCP CCP data connector

Simplify the development of connecting your GCP data source with a sample GCP CCP data connector deployment template.

GCP CCP example template

With most of the deployment template sections filled out, you only need to build the first two components, the output table and the DCR. For more information, see the Output table definition and Data Collection Rule (DCR) sections.

Data Connectors - Create or update

Reference the Create or Update operation in the REST API docs to find the latest stable or preview API version. The difference between the create and the update operation is the update requires the etag value.

PUT method

https://management.azure.com/subscriptions/{{subscriptionId}}/resourceGroups/{{resourceGroupName}}/providers/Microsoft.OperationalInsights/workspaces/{{workspaceName}}/providers/Microsoft.SecurityInsights/dataConnectors/{{dataConnectorId}}?api-version={{apiVersion}}

URI parameters

For more information about the latest API version, see Data Connectors - Create or Update URI Parameters.

Name	Description
dataConnectorId	The data connector ID must be a unique name and is the same as the `name` parameter in the request body.
resourceGroupName	The name of the resource group, not case sensitive.
subscriptionId	The ID of the target subscription.
workspaceName	The name of the workspace, not the ID. Regex pattern: `^[A-Za-z0-9][A-Za-z0-9-]+[A-Za-z0-9]$`
api-version	The API version to use for this operation.

Request body

The request body for a GCP CCP data connector has the following structure:

{
   "name": "{{dataConnectorId}}",
   "kind": "GCP",
   "etag": "",
   "properties": {
        "connectorDefinitionName": "",
        "auth": {},
        "request": {},
        "dcrConfig": ""
   }
}

GCP

GCP represents a CCP data connector where the paging and expected response payloads for your Google Cloud Platform (GCP) data source has already been configured. Configuring your GCP service to send data to a GCP Pub/Sub must be done separately. For more information, see Publish message in Pub/Sub overview.

Name	Required	Type	Description
name	True	string	The unique name of the connection matching the URI parameter
kind	True	string	Must be `GCP`
etag		GUID	Leave empty for creation of new connectors. For update operations, the etag must match the existing connector's etag (GUID).
properties.connectorDefinitionName		string	The name of the DataConnectorDefinition resource that defines the UI configuration of the data connector. For more information, see Data Connector Definition.
properties.auth	True	Nested JSON	Describes the credentials for polling the GCP data. For more information, see authentication configuration.
properties.request	True	Nested JSON	Describes the GCP project Id and GCP subscription for polling the data. For more information, see request configuration.
properties.dcrConfig		Nested JSON	Required parameters when the data is sent to a Data Collection Rule (DCR). For more information, see DCR configuration.

Authentication configuration

Authentication to GCP from Microsoft Sentinel uses a GCP Pub/Sub. You must configure the authentication separately. Use the Terraform scripts here. For more information, see GCP Pub/Sub authentication from another cloud provider.

As a best practice, use parameters in the auth section instead of hard-coding credentials. For more information, see Secure confidential input.

In order to create the deployment template which also uses parameters, you need to escape the parameters in this section with an extra starting [. This allows the parameters to assign a value based on the user interaction with the connector. For more information, see Template expressions escape characters.

To enable the credentials to be entered from the UI, the connectorUIConfig section requires instructions with the desired parameters. For more information, see Data connector definitions reference for the Codeless Connector Platform.

GCP auth example:

"auth": {
    "serviceAccountEmail": "[[parameters('GCPServiceAccountEmail')]",
    "projectNumber": "[[parameters('GCPProjectNumber')]",
    "workloadIdentityProviderId": "[[parameters('GCPWorkloadIdentityProviderId')]"
}

Request configuration

The request section requires the projectId and subscriptionNames from the GCP Pub/Sub.

GCP request example:

"request": {
    "projectId": "[[parameters('GCPProjectId')]",
    "subscriptionNames": [
        "[[parameters('GCPSubscriptionName')]"
    ]
}

DCR configuration

Field	Required	Type	Description
DataCollectionEndpoint	True	String	DCE (Data Collection Endpoint) for example: `https://example.ingest.monitor.azure.com`.
DataCollectionRuleImmutableId	True	String	The DCR immutable ID. Find it by viewing the DCR creation response or using the DCR API
StreamName	True	string	This value is the `streamDeclaration` defined in the DCR (prefix must begin with Custom-)

Example CCP data connector

Here's an example of all the components of the GCP CCP data connector JSON together.

{
    "kind": "GCP",
    "properties": {
        "connectorDefinitionName": "[[parameters('connectorDefinitionName')]",
        "dcrConfig": {
            "streamName": "[variables('streamName')]",
            "dataCollectionEndpoint": "[[parameters('dcrConfig').dataCollectionEndpoint]",
            "dataCollectionRuleImmutableId": "[[parameters('dcrConfig').dataCollectionRuleImmutableId]"
        },
    "dataType": "[variables('dataType')]",
    "auth": {
        "serviceAccountEmail": "[[parameters('GCPServiceAccountEmail')]",
        "projectNumber": "[[parameters('GCPProjectNumber')]",
        "workloadIdentityProviderId": "[[parameters('GCPWorkloadIdentityProviderId')]"
    },
    "request": {
        "projectId": "[[parameters('GCPProjectId')]",
        "subscriptionNames": [
            "[[parameters('GCPSubscriptionName')]"
            ]
        }
    }
}

For more information, see Create GCP data connector REST API example.

Share via