Ingest data from LinkedIn Ads

Important

This feature is in Beta. Workspace admins can control access to this feature from the Previews page. See Manage Azure Databricks previews.

Learn how to create a managed ingestion pipeline to ingest data from LinkedIn Ads into Azure Databricks.

Requirements

To create an ingestion pipeline, you must meet the following requirements:
- Your workspace must be enabled for Unity Catalog.
- Serverless compute must be enabled for your workspace. See Serverless compute requirements.
- To create a new connection, you must have CREATE CONNECTION privileges on the metastore. See Manage privileges in Unity Catalog.
  
  If the connector supports UI-based pipeline authoring, an admin can create the connection and the pipeline at the same time by completing the steps on this page. However, if the users who create pipelines use API-based pipeline authoring or are non-admin users, an admin must first create the connection in Catalog Explorer. See Connect to managed ingestion sources.
- To use an existing connection, you must have USE CONNECTION privileges or ALL PRIVILEGES on the connection object.
- You must have USE CATALOG privileges on the target catalog.
- You must have USE SCHEMA and CREATE TABLE privileges on an existing schema or CREATE SCHEMA privileges on the target catalog.
To ingest from LinkedIn Ads, you must complete the steps in Create a LinkedIn Ads connection.
You need the sponsored ad-account ID for each ad account you want to ingest. Eleven of the twelve source tables live in a per-account namespace named after that ID. See Source namespaces.

Create an ingestion pipeline

LinkedIn Ads supports API-based pipeline authoring only. Use Declarative Automation Bundles or the Pipelines REST API.

Declarative Automation Bundles

This tab describes how to deploy an ingestion pipeline using Declarative Automation Bundles. Bundles can contain YAML definitions of jobs and tasks, are managed using the Databricks CLI, and can be shared and run in different target workspaces (such as development, staging, and production). For more information, see What are Declarative Automation Bundles?.

Create a bundle using the Databricks CLI:
```
databricks bundle init
```
Add two new resource files to the bundle:
- A pipeline definition file (for example, resources/linkedin_ads_pipeline.yml).
- A job definition file that controls the frequency of data ingestion (for example, resources/linkedin_ads_job.yml).
See pipeline.ingestion_definition and Examples.
Deploy the pipeline using the Databricks CLI:
```
databricks bundle deploy
```

Pipelines REST API

Use the Pipelines API to create the pipeline. The ingestion_definition field takes a YAML specification. See Examples.

Examples

The following examples show YAML specifications that Declarative Automation Bundles or the REST API can use to create pipelines.

Ingest all five entity tables from one ad account

The account_history table comes from the default namespace. The other four entity tables come from the ad account's own namespace, so set source_schema to the sponsored ad-account ID.

resources:
  pipelines:
    pipeline_linkedin_ads:
      name: <pipeline-name>
      catalog: <destination-catalog>
      target: <destination-schema>
      ingestion_definition:
        connection_name: <connection-name>
        objects:
          - table:
              source_schema: default
              source_table: account_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
          - table:
              source_schema: <ad-account-id>
              source_table: campaign_group_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
          - table:
              source_schema: <ad-account-id>
              source_table: campaign_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
          - table:
              source_schema: <ad-account-id>
              source_table: creative_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
          - table:
              source_schema: <ad-account-id>
              source_table: account_user_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>

Ingest prebuilt reports with custom sync options

Each report takes its own settings from connector_options.linkedin_ads_options, which accepts sync_start_date (an ISO date string) and lookback_window_days (an integer from 0 to 365). Omit either key to take its default: a two-year start date and a seven-day lookback. Set the options per report, because each report tracks its own cursor.

All seven reports are per-account, so source_schema is always the sponsored ad-account ID.

resources:
  pipelines:
    pipeline_linkedin_ads_reports:
      name: <pipeline-name>
      catalog: <destination-catalog>
      target: <destination-schema>
      ingestion_definition:
        connection_name: <connection-name>
        objects:
          # Daily campaign report: backfill from an explicit date and widen the
          # lookback to 30 days so late-attributed conversions are re-read.
          - table:
              source_schema: <ad-account-id>
              source_table: ad_analytics_by_campaign_report
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
              connector_options:
                linkedin_ads_options:
                  sync_start_date: '2026-01-01'
                  lookback_window_days: 30
          # Daily creative report: same start date, default seven-day lookback.
          - table:
              source_schema: <ad-account-id>
              source_table: ad_analytics_by_creative_report
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
              connector_options:
                linkedin_ads_options:
                  sync_start_date: '2026-01-01'
          # Monthly demographic report: the start date is aligned to the first of
          # its month, so 2026-05-15 fetches all of May 2026.
          - table:
              source_schema: <ad-account-id>
              source_table: monthly_ad_analytics_by_member_industry_report
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
              connector_options:
                linkedin_ads_options:
                  sync_start_date: '2026-05-15'
                  lookback_window_days: 45
          # Monthly demographic report with no options: defaults to a two-year
          # start date, capped by the two-year demographic retention horizon.
          - table:
              source_schema: <ad-account-id>
              source_table: monthly_ad_analytics_by_member_seniority_report
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>

Ingest from more than one ad account

Because the per-account tables live in a namespace named after the ad-account ID, ingest a second account by repeating the table definitions with a different source_schema. Give each destination table a distinct name so the two accounts don't collide, because Azure Databricks cannot ingest two tables with the same name in one pipeline.

resources:
  pipelines:
    pipeline_linkedin_ads_multi_account:
      name: <pipeline-name>
      catalog: <destination-catalog>
      target: <destination-schema>
      ingestion_definition:
        connection_name: <connection-name>
        objects:
          - table:
              source_schema: <first-ad-account-id>
              source_table: campaign_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
              destination_table: campaign_history_account_1
          - table:
              source_schema: <second-ad-account-id>
              source_table: campaign_history
              destination_catalog: <destination-catalog>
              destination_schema: <destination-schema>
              destination_table: campaign_history_account_2

Start, schedule, and set alerts on your pipeline

After the pipeline is created, revisit the Azure Databricks workspace, then click Jobs & Pipelines in the left pane.

The new pipeline appears in the list. Click the pipeline name to view its details.
On the pipeline details page, click Start to run the pipeline immediately. To run it on a schedule, click Schedule. For details, see Run a pipeline update.
To set alerts on the pipeline, use the job that schedules it. From the pipeline details page, click Schedule, then select one of your schedules to show the job details.
On the job details pane, under Job notifications, set up notifications. See Add notifications on a job.
Monitor the pipeline update on the pipeline details page. After the update completes successfully, query your destination tables to confirm the data arrived.

Additional resources

Feedback

Was this page helpful?

Last updated on 2026-08-17