Configure Gmail for Azure Databricks ingestion

Learn how to configure a Google Cloud service account so that Azure Databricks can ingest data from your Gmail mailbox.

Important

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.

The Gmail connector authenticates with a Google service account that impersonates a single mailbox using domain-wide delegation. In this task, you create the service account and a JSON key, grant it read-only access to Gmail data, and identify the mailbox to impersonate. You'll use the authentication details you obtain to create a Azure Databricks Unity Catalog connection to Gmail.

Prerequisites

  • Access to the Google Cloud project where you want to create the service account, with permission to create service accounts and keys.
  • Google Workspace super administrator access to grant domain-wide delegation.
  • The email address of the Gmail mailbox that you want to ingest.

Create a service account and a JSON key

  1. In the Google Cloud console, go to IAM & Admin > Service Accounts.

  2. Click Create service account.

  3. Enter a name and description for the service account, then click Create and continue.

  4. Click Done. You don't need to grant project-level roles.

  5. On the Service Accounts page, click the service account that you created.

  6. On the Keys tab, click Add key > Create new key.

  7. Select JSON, then click Create. The JSON key file downloads to your machine.

    Store the JSON key securely. You paste its contents when you create the Unity Catalog connection.

Enable the Gmail API

  1. In the Google Cloud console, go to APIs & Services > Library.
  2. Search for Gmail API, then click it.
  3. Click Enable.

Grant domain-wide delegation

To read mailbox data on behalf of a user, the service account must be authorized for domain-wide delegation in your Google Workspace.

  1. In the Google Cloud console, copy the service account's Unique ID (client ID) from the Details tab.

  2. In the Google Admin Console, go to Security > Access and data control > API controls.

  3. Under Domain-wide delegation, click Manage domain-wide delegation.

  4. Click Add new.

  5. In Client ID, paste the service account's unique ID.

  6. In OAuth scopes, enter the following read-only scope:

    https://www.googleapis.com/auth/gmail.readonly
    
  7. Click Authorize.

Identify the mailbox to impersonate

The service account impersonates a single mailbox. Note the email address of the mailbox that you want to ingest, for example user@example.com. You provide this address as the impersonation email when you create the Unity Catalog connection.

To ingest more than one mailbox, create a separate connection and pipeline for each mailbox.

Next step

Create a Gmail connection