A/B Testing API

Warning

Deprecation Notice
The Marketing version 202209 (Marketing September 2022) and below has been sunset and the unversioned APIs are going to be sunset soon. We recommend that you migrate to the versioned APIs as well as migrate to the new Content and Community Management APIs to avoid disruptions. See the Migration page for more details. If you haven’t yet migrated and have questions, submit a request on the LinkedIn Developer Support Portal.

The A/B Testing API measures the performance of two campaigns with equal audiences and budgets to test one variable, such as creative content / ad format, targeting, or placement. The testing tool then selects a winner (if any), or verifies that the performance difference between the variants is insignificant or inconclusive. You can optimize for the following KPIs:

• Cost Per Click (CPC)
• Cost Per Thousand Impressions (CPM)
• Cost Per Video View (CPV)
• Cost Per Message Sent (CPMS)

Note

Currently, A/B Testing experiments can be created to compare 2 campaigns.

Experiments can be configured to automatically stop based on predefined criteria. For example, the experiment can be automatically stopped if a winner is picked at a defined confidence level. The confidence level can be defined when you create the experiment. You can also automatically stop the experiment based on a stop time that you define.

Members within the user’s target audience are randomly selected to see either variant of an experiment. Once selected, they will never see the other variant for the duration of the experiment. You can direct traffic unevenly by defining the trafficWeight field.

Important

All API calls documented here require the header for the latest Rest.li protocol: X-RestLi-Protocol-Version: 2.0.0. Refer to the Protocol Versions page to learn more about Rest.li Protocol 2.0.0.

Permissions

Permission	Description
rw_ads	Manage and read an authenticated member's experiments. Restricted to experiments belonging to ad accounts in which the authenticated member has one of the following ad account roles. ACCOUNT_BILLING_ADMIN ACCOUNT_MANAGER CAMPAIGN_MANAGER
r_ads	Read an authenticated member's ad accounts. Restricted to experiments belonging to ad accounts in which the authenticated member has one of the following ad account roles. ACCOUNT_BILLING_ADMIN ACCOUNT_MANAGER CAMPAIGN_MANAGER CREATIVE_MANAGER VIEWER

See Account Access Controls for more information on ad account roles.

Note

If you are using legacy permissions, please refer to the Marketing Permissions Migration page for requesting legacy permissions.

Best Practices

Although we cannot guarantee that your experiment will be conclusive — i.e., we cannot guarantee that we will be able to either (A) identify a winner or (B) confirm that there is no winner. However, we have provided a series of best practices that may increase your chances of getting a conclusive test result.

Check for Experiment Warnings

Validation on potential experiment configuration issues can be pulled through the validateExperimentSetup Rest.li action. These warnings aim to alert you of issues that may compromise the results of your experiment.

Warnings can be fetched even before experiment is saved. We recommend always checking validation setup and addressing any warnings before the experiment is saved. The following flow is recommended to manage warnings:

1. Create or edit an experiment setup.
2. Call validateExperimentSetup Rest.li action and fetch the experiment’s validation result for any errors/warnings.
3. Resolve any warnings that is unintended in experiment setup, and then save or update the experiment.

Set a High Confidence Level

While we permit setting confidence levels as low as 51%, the best practice is to use higher confidence levels. We recommend confidence levels of at least 90%.

Schedule Long Running Experiments

We allow experiments to run for up to 90 days. The longer an experiment is run, the more conclusive and reliable the results will be.

Set Proportional Budgets

An experiment has the most power if the campaigns that it is comparing are weighted equally. The more you skew the traffic weight, the less power you’ll have. To ensure that you're weighted equally, we recommend that you create two new campaigns for your A/B test. However, if you do leverage uneven traffic weight, then the campaign’s budgets should be proportional to their weights.

Restrictions

We prohibit creating experiments of the following types:

• Experiments that have the goal of manipulating / altering member’s moods or emotions.
• Experiments designed to intentionally deliver a negative member experience.
• Experiments that involve sensitive categories, including race, ethnicity, national origin, gender, health, age, religion, political opinions, sexual orientation and union membership.

Schemas

AdExperiment Schema

Field	Type	Description
account	sponsoredAccountUrn	The sponsoredAccount URN identifying the owner of the experiment. This field is create only.
autoStop	String	An enum describing the configuration on when to stop the experiment and its campaigns. Possible values are: AUTO_STOP_AT_END_DATE - Default value. Stop the experiment once it reaches its end date, and automatically pause all campaigns within the experiment. AUTO_STOP_AT_WINNER - Stop the experiment once a winner has been declared or the experiment reaches its end date, and automatically pause all campaigns within the experiment. AUTO_STOP_DISABLED - Stop the experiment once it reaches its end date, but allow the campaigns to keep running. The traffic split no longer applies to the campaigns after the experiment is stopped.
cells	Array of AdExperimentCells	A list of cells to compare under the experiment. This field must contain 2 cells. The array isn’t editable after the experiment has started, the cell names and descriptions can be updated.
confidence	integer	An integer percent defining the confidence level of the experiment. For example, a value of 90 requires the experiment result to reach a confidence level of 90% to be considered valid.
description	String	The description of this experiment. This is an optional field. Its length must not be more than 1000 characters.
endedAt	Epoch timestamp in milliseconds	The experiment’s actual end time. NULL until the experiment ends. The system may terminate the experiment before the scheduled end time. This field is read only.
executionState	String	An enum describing the execution state of the experiment. This field is read only. Possible values are: RUNNING - Experiment is running. PENDING - Experiment will start in the future, for example, the user has created the experiment, but the start date is set to future. ON_HOLD - Experiment is temporarily suspended. STOPPED - Experiment is stopped. However, it can run again in the future, for example, user extends the scheduledEndAt field. ENDED - Experiment is ended. It becomes immutable and cannot run again.
id	long	The ID uniquely identifying the experiment. This field is read only.
keyMetric	String	A single metric defining the winning criteria - usually a cost-per-action metric such as cost-per-click. Possible values are: CPC - Cost per click CPM - Cost per thousand impressions CPV - Cost per video view CPMS - Cost per message sent
name	String	The name of this experiment. Does not have to be unique in account. This is a mandatory field. Its length must not be more than 100 characters.
scheduledEndAt	Epoch timestamp in milliseconds	The experiment’s end time, as suggested by the user. Should be greater than the start time. Also, experiment cannot be set to run for more than 90 days (i.e. end time - start time <= 90 days).
scheduledStartAt	Epoch timestamp in milliseconds	The experiment’s start time, as suggested by the user. Setting a start time in the past will cause the experiment to start right away.
terminationReason	String	A message explaining why the experiment was moved to an ENDED state by LinkedIn. This field is read only. Our system will automatically end experiments for the following reasons: It has been 90 days since the experiment’s start date. The experiment hit an erroneous condition.
userIntendedState	String	An enum describing the available states for the user to select in order to control the experiment. Possible values are: ACTIVE CANCELED DRAFT PAUSED

AdExperimentCell Schema

An AdExperimentCell contains a reference to an existing ads entity (e.g. a sponsoredCampaign URN) to be used in the experiment. It also contains metadata, such as the cell name and what percentage of traffic should see this cell.

Field	Type	Description
name	String	The name of this cell. Does not need to be unique. This is a mandatory field. Its length must be less than 100 characters.
description	String	A memo of what this cell is about. This is an optional field. Its length must be less than 500 characters.
reference.campaign	SponsoredCampaignUrn	The ads entity that this cell is associated with.
trafficWeight	integer	An integer defining the percentage of traffic that will receive the cell treatment. The sum of all traffic weights in the experiment must equal 100.

Create Experiments

New experiments can be created by issuing a POST to the below URL. Experiments can be created individually.

POST https://api.linkedin.com/rest/adExperiments

User must have one of the following ad account permissions in order to create an experiment:

• ACCOUNT_BILLING_ADMIN
• ACCOUNT_MANAGER
• CAMPAIGN_MANAGER

Note

An experiment must meet the following conditions. Otherwise, it will be rejected.

• All required fields are present.
• The start date is before the end date.
• The experiment run time (i.e., end date - start date) is less than or equal to 90 days.
• The confidence level is between 51 and 99.
• Each individual cell’s trafficWeight value is between 1 and 99.
• The sum of all cells’ trafficWeight values is equal to 100.
• Any campaign included in a cell is not part of any other active or future experiments with overlapping run dates.
• The experiment includes 2 cells with distinct campaigns from the same campaign group (and, therefore, from the same account).

Before creation, you may check the validity of your experiment’s configuration by using the validation API. This endpoint will return warnings and errors indicating when something in your experiment may skew the results. Please refer to Validate Experiments for more information on warnings and errors that you may encounter.

Sample Request

POST https://api.linkedin.com/rest/adExperiments

{
  "name": "Experiment 1",
  "account": "urn:li:sponsoredAccount:123",
  "keyMetric": "CPC",
  "confidence": 90,
  "scheduledEndAt": 1562964400000,
  "scheduledStartAt": 1561964400000,
  "userIntendedState": "ACTIVE",
  "autoStop": "AUTO_STOP_DISABLED",
  "cells": [
    {
      "name": "cell1",
      "trafficWeight": 50,
      "reference": {
        "campaign": "urn:li:sponsoredCampaign:123"
      }
    },
    {
      "name": "cell2",
      "trafficWeight": 50,
      "reference": {
        "campaign": "urn:li:sponsoredCampaign:345"
      }
    }
  ]
}

Successful creation requests will return a 201 status with a full response body of the successfully created experiment along with its id.

Exceptions

Following are expected exceptions from the Create end point.

Code	HTTP Status	Message	Error Detail Type
ACCOUNT_NOT_FOUND	422	The ad account does not exist.	AdExperimentValidationDetail
ACCOUNT_ALREADY_RUNNING_BRAND_LIFT_TEST	422	Account provided has a brand lift test with overlapping schedule. A/B test cannot run concurrently with a brand lift test.	AdExperimentValidationDetail
CAMPAIGN_ALREADY_SCHEDULED	409	There is an already existing experiment with overlapping schedule containing same campaign.	AdExperimentValidationDetail
CAMPAIGN_BID_MUST_BE_GREATER_THAN_MINIMUM	422	You must set a higher bid price than minimum for this campaign type.	AdExperimentValidationDetail
CAMPAIGN_GROUPS_DIFFERENT	422	The campaigns in the experiment belong to different campaign groups under the same account.	AdExperimentValidationDetail
CAMPAIGN_GROUP_DYNAMIC_BUDGET_OPTIMIZATION_MUST_BE_DISABLED	422	You should disable Campaign Group Dynamic Budget Optimization for valid A/B test.	AdExperimentValidationDetail
CAMPAIGN_NOT_FOUND	422	The campaign does not exist.	AdExperimentValidationDetail
CAMPAIGN_NOT_SERVABLE	422	System does not allow campaign to be served. For example, campaign is canceled. This cannot be reverted by the user.	AdExperimentValidationDetail
CAMPAIGNS_FROM_DIFFERENT_ACCOUNT	422	The campaigns referenced in the experiment belong to two separate accounts.	AdExperimentValidationDetail
DUPLICATE_ELEMENTS_IN_CELLS	422	Both cells reference the same entity.	AdExperimentValidationDetail
FORBIDDEN	403	User has no access.
INVALID_CONFIDENCE_VALUE	422	Experiment’s confidence value must be >= 51 AND <= 99.	AdExperimentValidationDetail
INVALID_END_TIME	422	Scheduled end time is not valid. It cannot be in the past or more than 180 days in the future.	AdExperimentValidationDetail
INVALID_NO_OF_CELLS	422	The experiment must contain exactly 2 cells.	AdExperimentValidationDetail
INVALID_SCHEDULE	422	Experiment’s schedule is not valid. Experiment cannot be scheduled to run for more than 90 days.	AdExperimentValidationDetail
INVALID_START_TIME	422	Scheduled start time is not valid. It has to be positive.	AdExperimentValidationDetail
INVALID_STATE_TRANSITION	422	The user is attempting to make a state change that is not allowed.	AdExperimentValidationDetail
INVALID_SUM_OF_WEIGHT_OF_CELLS	422	The sum of traffic weight of cells must equal 100.	AdExperimentValidationDetail
INVALID_WEIGHT_OF_CELL	422	A cell has an unexpected traffic weight. It must be >= 1, and <= 99.	AdExperimentValidationDetail
MULTIPLE_VALIDATIONS_FAILED	422	Multiple errors occurred during the input validation. Please see errorDetails for more information.	AdExperimentErrorDetail
SCHEMA_MISMATCH	422	Input does not match schema.	AdExperimentValidationDetail
SET_OR_UPDATE_NOT_ALLOWED	422	The request is attempting to either set or modify a field(s) which is not allowed.	AdExperimentValidationDetail

Validate Experiments

Experiments can be validated for potential configuration issues that could compromise the results of the experiment. For example, if a Sponsored Campaign in an experiment was running while the experiment was on hold, this could skew the experiment results.

The validation status can be retrieved at any time during and after the creation of an experiment.

Possible Validation Warnings

For the full list of possible validation warnings and errors, see Validation Error Code.

AdExperimentValidationResult Schema

Field	Type	Description
experimentId	long	The ID of the experiment.
items	Array of AdExperimentValidationResultItems	An array containing all of the validation result items.
validatedAt	Epoch timestamp in milliseconds	The time when the validation was run.

AdExperimentValidationResultItem Schema

Field	Type	Description
code	String	The error code of the validation result item.
detail	AdExperimentValidationDetail	Additional data associated with a single validation error of experiment.
level	String	An enum describing the level of a validation result item. Possible values: WARNING ERROR
message	String	The message of the validation result item.

AdExperimentValidationDetail Schema

Field	Type	Description
associatedEntity	SponsoredCampaignUrn, SponsoredAccountUrn	Urn of the entity because of which validation error happened.
pathErrors	array of InputError	A list of InputError where each InputError contains information about the pathSpec and error in it

Validate Endpoint

These warnings are retrievable through a sub-resource to check the validity of an experiment.

POST https://api.linkedin.com/rest/adExperiments?action=validateExperimentSetup

This endpoint returns an AdExperimentValidationResult object. This object contains an array of objects called AdExperimentValidationResultItem that provide details on what may be misconfigured. The schemas of these 2 objects are defined above.

Important

When a new experiment (which is not stored) is sent for validation, the id field cannot be present in the request body. When an existing entity or modified version of existing entity is sent for validation, id must be present in the request body.

Sample Request

POST https://api.linkedin.com/rest/adExperiments?action=validateExperimentSetup

{
  "adExperiment": {
    "name": "Test1",
    "account": "urn:li:sponsoredAccount:1234567",
    "keyMetric": "CPC",
    "confidence": 90,
    "scheduledEndAt": 1562119781530,
    "scheduledStartAt": 1561119781530,
    "userIntendedState": "ACTIVE",
    "autoStop": "AUTO_STOP_DISABLED",
    "cells": [
      {
        "name": "cell1",
        "trafficWeight": 50,
        "reference": {
          "campaign": "urn:li:sponsoredCampaign:12345"
        }
      },
      {
        "name": "cell2",
        "trafficWeight": 50,
        "reference": {
          "campaign": "urn:li:sponsoredCampaign:22345"
        }
      }
    ]
  }
}

Sample Response

{
  "value": {
    "items": [
      {
        "code": "CAMPAIGN_STARTS_TOO_EARLY",
        "level": "WARNING",
        "detail": {
          "associatedEntity": {
            "campaign": "urn:li:sponsoredCampaign:12345"
          }
        },
        "message": "The campaign is scheduled to run before the experiment scheduled start time."
      }
    ],
    "validatedAt": 1557857848554
  }
}

Validation Error Code

Error	Description	Level	Impact
AB_TEST_BUDGET_BELOW_RECOMMENDATION	While conclusive results are never guaranteed, you can increase the likelihood of a conclusive test result by increasing your test budget to $5,000	WARN
AB_TEST_CANCELED	Due to your particular test setup, your test results and statistical strength can continue to fluctuate up or down for a number of days after your scheduled end date. For the most accurate readout, wait until your test has completed and this period has passed.	WARN
AB_TEST_DURATION_BELOW_RECOMMENDATION	While conclusive results are never guaranteed, you can increase the likelihood of a conclusive test result by running your test for at least 3 weeks.	WARN
AB_TEST_DURATION_BELOW_REQUIREMENT	Each variant in the test must have an audience size of at least 1,000,000.	ERROR	An experiment cannot be created
AB_TEST_PAUSED	Pausing this test will pause both campaigns. Your test's end date will not change.	WARN
ACCOUNT_ALREADY_RUNNING_BRAND_LIFT_TEST	Account provided has a brand lift test with overlapping schedule. A/B test cannot run concurrently with a brand lift test.	ERROR	An experiment cannot be created
ACCOUNT_NOT_FOUND	The ad account does not exist.	ERROR	Ad experiment cannot be updated
ACCOUNT_NOT_SERVABLE	System does not allow account to be served. For example, account is canceled. This cannot be reverted by user.	ERROR	Ad experiment cannot be updated
ACCOUNT_ON_HOLD	User does not allow account to be served. For example, account has total budget on hold flag. This can be reverted by user.	WARN	May result in experiment being put on hold while it runs; experiment may not run for its intended duration
ACCOUNT_TOTAL_BUDGET_BELOW_AB_TEST_BUDGET	The ad account's budget doesn't cover this A/B test.	ERROR	An experiment cannot be created
ACCOUNT_TOTAL_BUDGET_ENDS_TOO_SOON	The ad account’s total budget ends before the experiment ends.	WARN	May result in experiment being put on hold while it runs; experiment may not run for its intended duration
BUDGET_AND_CELL_WEIGHT_NOT_PROPORTIONAL	The budget allocation of the cells is not proportional to their traffic weight allocation.	WARN	May result in biased experiment result.
CAMPAIGN_ALREADY_SCHEDULED	There is an already existing experiment with overlapping schedule containing same campaign.	ERROR	Ad experiment cannot be updated.
CAMPAIGN_AUDIENCE_EXPANSION_ENABLED	The campaign has the Audience Expansion feature enabled. A/B testing is currently not supported for the Audience Expansion feature, and this may result in inaccurate statistical results.	WARN	May result in biased experiment result
CAMPAIGN_AUDIENCE_SIZE_BELOW_RECOMMENDATION	While conclusive results are never guaranteed, you can increase the likelihood of a conclusive test result by increasing your test budget to $5,000.	WARN
CAMPAIGN_AUDIENCE_SIZE_BELOW_REQUIREMENT	Each variant in the test must have an audience size of at least 1,000,000.	ERROR	An experiment cannot be created.
CAMPAIGN_BID_MUST_BE_GREATER_THAN_MINIMUM	You must set a higher bid price than minimum for this campaign type.	ERROR	An experiment cannot be created.
CAMPAIGN_DIFFERENT_TARGETING	The campaigns referenced in the experiment are targeting different audiences.	WARN
CAMPAIGN_FINISHES_TOO_EARLY	The campaign is scheduled to end before the experiments scheduled end time.	WARN	May result in experiment being put on hold while it runs; experiment may not run for its intended duration
CAMPAIGN_GROUP_DYNAMIC_BUDGET_OPTIMIZATION_MUST_BE_DISABLED	You should disable Campaign Group Dynamic Budget Optimization for valid A/B test.	ERROR	An experiment cannot be created.
CAMPAIGN_GROUP_SCHEDULE_ENDS_TOO_SOON	Your campaign group's budget ends before the test end date. If you don't update the campaign group budget, then the test may be put on hold before it reaches its end date.	WARN
CAMPAIGN_GROUP_TOTAL_BUDGET_BELOW_AB_TEST_BUDGET	The campaign group's budget doesn't cover this A/B test.	ERROR	An experiment cannot be created
CAMPAIGN_GROUPS_DIFFERENT	The campaigns in the experiment belong to different campaign groups under the same account.	ERROR	Ad experiment cannot be updated
CAMPAIGN_NOT_FOUND	The campaign does not exist.	ERROR	Ad experiment cannot be updated
CAMPAIGN_NOT_SERVABLE	System doesn’t allow campaign to be served. For example, campaign is canceled. This cannot be reverted by the user.	ERROR	Ad experiment cannot be updated
CAMPAIGN_ON_HOLD	User does not allow campaign to be served. For example, campaign is paused by user. This can be reverted by user.	WARN	May result in experiment being put on hold while it runs; experiment may not run for its intended duration
CAMPAIGN_STARTS_TOO_EARLY	The campaign is scheduled to run before the experiment starts.	WARN	May result in biased experiment result
CAMPAIGN_STARTS_TOO_LATE	The campaign is scheduled to start after the experiments scheduled start time.	WARN	May result in experiment being put on hold while it runs; experiment may not run for its intended duration
CAMPAIGNS_FROM_DIFFERENT_ACCOUNT	Campaigns referenced in the experiment belong to two separate accounts.	ERROR	Ad experiment cannot be updated
CREATIVE_NOT_SERVABLE	System does not allow any creatives from one of the cells to be served. For example, all creatives are canceled in one campaign. This cannot be reverted by user.	ERROR	Will result in experiment being marked as canceled before its result is reached
CREATIVE_ON_HOLD	User does not allow any creatives from one of cells to be served. For example, all creatives are paused in one campaign. This can be reverted by user.	WARN	May result in experiment being put on hold while it runs; experiment may not run for its intended duration.
DUPLICATE_ELEMENTS_IN_CELLS	Both cells reference the same entity.	ERROR	Ad experiment cannot be updated
INVALID_CONFIDENCE_VALUE	Experiment’s confidence value must be >= 51 AND <= 99.	ERROR	Ad experiment cannot be updated
INVALID_END_TIME	Scheduled end time is not valid. It cannot be in the past or more than 180 days in the future.	ERROR	Ad experiment cannot be updated
INVALID_KEY_METRIC	The campaigns cannot provide information for the experiment's KeyMetric. For example, campaign ad format is not video, but experiment's KeyMetric is CPV.	ERROR	Experiment cannot be created
INVALID_NO_OF_CELLS	The experiment must contain exactly 2 cells.	ERROR	Ad experiment cannot be updated
INVALID_SCHEDULE	Experiment’s schedule is not valid. Experiment cannot be scheduled to run for more than 90 days.	ERROR	Ad experiment cannot be updated
INVALID_START_TIME	Scheduled start time is not valid. It has to be positive.	ERROR	Ad experiment cannot be updated
INVALID_STATE_TRANSITION	The user is attempting to make a state change that is not allowed.	ERROR	Ad experiment cannot be updated
INVALID_SUM_OF_WEIGHT_OF_CELLS	The sum of traffic weight of cells must equal 100.	ERROR	Ad experiment cannot be updated
INVALID_WEIGHT_OF_CELL	A cell has an unexpected traffic weight. Cell weights must be >= 1, and <= 99.	ERROR	Ad experiment cannot be updated
NO_ACTIVE_CREATIVES	At least one variant in your A/B test has no Active creatives. Your A/B test can't run until both variants have at least one Active creative.	ERROR
SCHEMA_MISMATCH	Input does not match the schema.	ERROR	Ad experiment cannot be updated
SET_OR_UPDATE_NOT_ALLOWED	The request is attempting to either set or modify a field(s) which is not allowed.	ERROR	Ad experiment cannot be updated
SOME_CREATIVES_IN_DRAFT	At least one creative in your A/B test is in Draft. A creative can't run as part of your A/B test until it is Active.	WARN

Exceptions

Valid Rest.li action should always succeed without any exception.

Get Experiment

Experiments can be fetched individually. Use the endpoint below to fetch a single experiment.

GET https://api.linkedin.com/rest/adExperiments/{AdExperimentId}

Sample Response

{
  "id": 123,
  "account": "urn:li:sponsoredAccount:123",
  "name": "Test Experiment",
  "description": "This is a test experiment",
  "scheduledStartAt": 1542148557000,
  "scheduledEndAt": 1544740556000,
  "endedAt": 1544740556000,
  "cells": [
    {
      "name": "Campaign 1",
      "description": "This is campaign 1",
      "trafficWeight": 50,
      "reference": {
        "campaign": "urn:li:sponsoredCampaign:123"
      }
    },
    {
      "name": "Campaign 2",
      "description": "This is campaign 2",
      "trafficWeight": 50,
      "reference": {
        "campaign": "urn:li:sponsoredCampaign:345"
      }
    }
  ],
  "userIntendedState": "ACTIVE",
  "executionState": "ENDED",
  "autoStop": "AUTO_STOP_AT_END_DATE",
  "confidence": 90,
  "keyMetric": "CPC"
}

Exceptions

Following are expected exceptions from the Get end point:

Code	HTTP Status	Message
EXPERIMENT_NOT_FOUND	404	AdExperiment referenced in the request doesn't exist.

Batch Get Experiments

Experiments can be fetched in batch as below.

GET https://api.linkedin.com/rest/adExperiments?ids=List(12685,11111)

Sample Response

{
  "results": {
    "12685": {
      "keyMetric": "CPM",
      "scheduledStartAt": 1565568000000,
      "scheduledEndAt": 1573171200000,
      "created": {
        "actor": "urn:li:member:123456",
        "time": 1565631108588
      },
      "confidence": 90,
      "executionState": "ON_HOLD",
      "autoStop": "AUTO_STOP_AT_WINNER",
      "userIntendedState": "ACTIVE",
      "cells": [
        {
          "name": "control",
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:123456"
          },
          "trafficWeight": 50
        },
        {
          "name": "test",
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:223456"
          },
          "trafficWeight": 50
        }
      ],
      "name": "My Test",
      "id": 12685,
      "lastModified": {
        "actor": "urn:li:member:123456",
        "time": 1565631108594
      },
      "account": "urn:li:sponsoredAccount:123456"
    }
  },
  "errors": {
    "11111": {
      "code": "EXPERIMENT_NOT_FOUND",
      "exceptionClass": "com.linkedin.restli.server.RestLiServiceException",
      "message": "AdExperiment referenced in the request doesn't exist",
      "status": 404
    }
  }
}

Exceptions

Batch get should always expect 200 response code. Each experiment is mapped to an individual response.

Update Experiments

Experiments can be updated before and after the experiment starts.

POST https://api.linkedin.com/rest/adExperiments/{AdExperimentId}

Before the experiment starts, nearly all fields can be updated. The only exception is account which can never be modified.

After the experiment starts, only the following fields can be updated:

• name
• description
• scheduledEndAt
• autoStop
• confidence
• userIntendedState
• AdExperimentCell.name
• AdExperimentCell.description

While scheduledEndAt can be updated after the experiment has already started, it cannot be extended so far that scheduledEndAt - scheduledStartAt > 90 days.

All fields not included in the above list are immutable once the experiment starts.

Once the experiment is ended, all fields are immutable.

The user must have one of the following ad account permissions in order to update an experiment.

• ACCOUNT_BILLING_ADMIN
• ACCOUNT_MANAGER
• CAMPAIGN_MANAGER

The endpoint for updating a single experiment is below. The request must include this header: X-RestLi-Method: partial_update.

The patch content in the request body must contain the entire top level field being updated. This is especially important if the field is a complex object. For example, to change the description of an experiment cell, the request must contain the entire body of the cells field, rather than just a single field of a single cell.

Sample Request

POST https://api.linkedin.com/rest/adExperiments/1234

{
  "patch": {
    "$set": {
      "scheduledEndAt": 1792221412
    }
  }
}

Sample Request

The following example updates a value within the cells array so the entire cells array must be provided.

POST https://api.linkedin.com/rest/adExperiments/1234

{
  "patch": {
    "$set": {
      "cells": [
        {
          "name": "Updated Campaign 1",
          "description": "This is campaign 1",
          "trafficWeight": 50,
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:123"
          }
        },
        {
          "name": "Campaign 2",
          "description": "This is campaign 2",
          "trafficWeight": 50,
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:345"
          }
        }
      ]
    }
  }
}

Exceptions

Following are expected exceptions from Update end point.

Code	Http Status	Message	Error Detail Type
ACCOUNT_ALREADY_RUNNING_BRAND_LIFT_TEST	422	Account provided has a brand lift test with overlapping schedule. A/B test cannot run concurrently with a brand lift test.	AdExperimentValidationDetail
CAMPAIGN_ALREADY_SCHEDULED	409	There is an already existing experiment with overlapping schedule containing same campaign.	AdExperimentValidationDetail
CAMPAIGN_GROUPS_DIFFERENT	422	The campaigns in the experiment belong to different campaign groups under the same account.	AdExperimentValidationDetail
CAMPAIGN_NOT_FOUND	422	The campaign does not exist.	AdExperimentValidationDetail
CAMPAIGN_NOT_SERVABLE	422	System does not allow campaign to be served. For example, campaign is canceled. This cannot be reverted by the user.	AdExperimentValidationDetail
CAMPAIGNS_FROM_DIFFERENT_ACCOUNT	422	The campaigns referenced in the experiment belong to two separate accounts.	AdExperimentValidationDetail
DUPLICATE_ELEMENTS_IN_CELLS	422	Both cells reference the same entity.	AdExperimentValidationDetail
EXPERIMENT_NOT_FOUND	404	AdExperiment referenced in the request doesn't exist.
FORBIDDEN	403	User has no access.
INVALID_CONFIDENCE_VALUE	422	Experiment’s confidence value must be >= 51 AND <= 99.	AdExperimentValidationDetail
INVALID_END_TIME	422	Scheduled end time is not valid. It cannot be in the past or more than 180 days in the future.	AdExperimentValidationDetail
INVALID_NO_OF_CELLS	422	The experiment must contain exactly 2 cells.	AdExperimentValidationDetail
INVALID_SCHEDULE	422	Experiment’s schedule is not valid. Experiment cannot be scheduled to run for more than 90 days.	AdExperimentValidationDetail
INVALID_START_TIME	422	Scheduled start time is not valid. It has to be positive.	AdExperimentValidationDetail
INVALID_STATE_TRANSITION	422	The user is attempting to make a state change that is not allowed.	AdExperimentValidationDetail
INVALID_SUM_OF_WEIGHT_OF_CELLS	422	The sum of traffic weight of cells must equal 100.	AdExperimentValidationDetail
INVALID_WEIGHT_OF_CELL	422	A cell has an unexpected traffic weight. It must be >= 1, and <= 99.	AdExperimentValidationDetail
MULTIPLE_VALIDATIONS_FAILED	422	Multiple errors occurred during the input validation. Please see errorDetails for more information.	AdExperimentErrorDetail
SCHEMA_MISMATCH	422	Input does not match schema.	AdExperimentValidationDetail
SET_OR_UPDATE_NOT_ALLOWED	422	The request is attempting to either set or modify a field(s) which is not allowed.	AdExperimentValidationDetail
WRITE_CONFLICT	409	User is trying to modify an entity which is already updated.

Find Experiments

Experiments can be fetched in batch according to certain search criteria.

GET https://api.linkedin.com/rest/adExperiments?q=search&search=({criteria}:(values:List({value_list})))

Query Parameters

Parameter	Description	Format	Required
q	Designates the query finder. Must be search for the Experiment Finder.	String	Yes
search	Search criteria, the following enum values are supported: accountCriteria idCriteria campaignCriteria campaignGroupCriteria keyMetricCriteria userIntendedStateCriteria executionStateCriteria	String	Yes
sort	Optional sort criteria, contains a list of field and order	List	No
	field supports the following enum values: ID NAME SCHEDULED_START_AT SCHEDULED_END_AT ENDED_AT USER_INTENDED_STATE EXECUTION_STATE CREATED	String	Yes
	order supports the following enum values: ASCENDING DESCENDING If not provided, it defaults to ASCENDING.	String	No

Search Criteria	Description	Value Format	Required
accountCriteria	Search by Ad account	Account URN	No
campaignCriteria	Search by campaign	Sponsored Campaign URN	No
campaignGroupCriteria	Search by campaign group	Campaign Group URN	No
executionStateCriteria	Search by executionState of experiment, the following enum values are supported: PENDING RUNNING ON_HOLD STOPPED ENDED	String	No
idCriteria	Search by experiment id	Long	No
keyMetricCriteria	Search by key metric of experiment, the following enum values are supported: CPC: Cost per click CPM: Cost per thousand impressions CPV: Cost per video view CPMS: Cost per message sent	String	No
userIntendedStateCriteria	Search by userIntendedState of experiment, the following enum values are supported: ACTIVE CANCELED DRAFT PAUSED	String	No

Sample Request

The following sample request finds all experiments that contain a specific campaign 1234567 as its cell, sorting by name and created timestamp.

GET https://api.linkedin.com/rest/adExperiments?q=search&search=(campaignCriteria:(values:List(urn%3Ali%3AsponsoredCampaign%3A1234567)))&sort=List((field:NAME),(field:CREATED,order:DESCENDING))

The following is a complex sample request that involve all search criteria.

GET https://api.linkedin.com/rest/adExperiments?/adExperiments?q=search&search=(accountCriteria:(values:List(urn%3Ali%3AsponsoredAccount%3A1234567)),campaignCriteria:(values:List(urn%3Ali%3AsponsoredCampaign%3A1234567)),keyMetricCriteria:(values:List(CPC)),userIntendedStateCriteria:(values:List(ACTIVE)),executionStateCriteria:(values:List(PENDING,RUNNING)),campaignGroupCriteria:(values:List(urn%3Ali%3AsponsoredCampaignGroup%3A1234567)))

Sample Response

{
  "elements": [
    {
      "keyMetric": "CPC",
      "scheduledStartAt": 1568062327739,
      "scheduledEndAt": 1575162000000,
      "created": {
        "actor": "urn:li:member:1234567",
        "time": 1568063540121
      },
      "confidence": 90,
      "executionState": "RUNNING",
      "autoStop": "AUTO_STOP_DISABLED",
      "userIntendedState": "ACTIVE",
      "cells": [
        {
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:1234567"
          },
          "trafficWeight": 50,
          "name": "test cell"
        },
        {
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:2234567"
          },
          "trafficWeight": 50,
          "name": "control cell"
        }
      ],
      "name": "Test blue vs. red",
      "id": 11113,
      "lastModified": {
        "actor": "urn:li:member:1234567",
        "time": 1568063540121
      },
      "account": "urn:li:sponsoredAccount:1234567"
    },
    {
      "keyMetric": "CPC",
      "scheduledStartAt": 1568062327739,
      "scheduledEndAt": 1575162000000,
      "created": {
        "actor": "urn:li:member:1234567",
        "time": 1568062451794
      },
      "confidence": 90,
      "executionState": "ENDED",
      "autoStop": "AUTO_STOP_DISABLED",
      "userIntendedState": "CANCELED",
      "cells": [
        {
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:1234567"
          },
          "trafficWeight": 50,
          "name": "test cell"
        },
        {
          "reference": {
            "campaign": "urn:li:sponsoredCampaign:3234567"
          },
          "trafficWeight": 50,
          "name": "control cell"
        }
      ],
      "name": "Test blue vs. purple",
      "id": 11103,
      "lastModified": {
        "actor": "urn:li:member:1234567",
        "time": 1568063172152
      },
      "account": "urn:li:sponsoredAccount:1234567"
    }
  ],
  "paging": {
    "total": 2,
    "count": 10,
    "start": 0,
    "links": []
  }
}

Exceptions

Finder requests should always succeed without any exception.

Delete Experiments

An experiment can be deleted by issuing a DELETE call to the URL below. Experiments can only be deleted individually.

DELETE https://api.linkedin.com/rest/adExperiments/{AdExperimentId}

The user must have one of the following ad account permissions in order to delete an experiment:

• ACCOUNT_BILLING_ADMIN
• ACCOUNT_MANAGER
• CAMPAIGN_MANAGER

To delete an experiment, its executionState must be PENDING. If it’s not, then the request will be rejected with an HTTP response code 400.

Sample Request

DELETE https://api.linkedin.com/rest/adExperiments/1234

Exceptions

Following are expected Exception from Delete end point.

Code	HTTP Status	Message	Error Detail Type
DELETION_NOT_ALLOWED	405	Delete operation is not allowed except for experiments in PENDING execution state.
EXPERIMENT_NOT_FOUND	404	AdExperiment referenced in the request doesn't exist.
FORBIDDEN	403	User has no access.

Get Experiment Results

Experiment’s result contains details on the cell that won the experiment, as well as the confidence for the experiment result. This is re-evaluated on a daily basis, always folding in the new data collected over the past day. The generatedOn field is the cutoff point at which we stopped considering that new data.

A single result can be retrieved as below:

GET https://api.linkedin.com/rest/adExperimentResults/{AdExperimentId}

AdExperimentResult Schema

Field	Type	Description
experimentId	long	The ID of the experiment.
generatedOn	date	The cutoff time for considering data to generate the current experiment result. Always midnight UTC. Of the format (day, month, year).
pValue	double	The p value for the analysis result. In the range of [0, 1], with two digits of precision after the decimal point. Empty if no winner is selected. We only report the p value and the winner if (1 - p value) is greater than or equal to the confidence level specified at experiment setup.
minimumDetectableEffect	double	The minimum detectable effect is the smallest difference between two cells that the system can detect in an experiment. For example, a minimum detectable effect of 0.08 means that the system can only detect a difference if one cell performs at least 8% better than the other. It can also be used to better understand experiment results. When an experiment comes back as no winner but has a small minimum detectable effect (e.g., 0.02), then there is no practical difference between the two cells. We suggest using a threshold of 0.1 for this purpose.
winner	AdExperimentCell URN	The URN of the winning cell, if the system is able to identify a winner of the experiment.
cellResults	Array of AdExperimentCellResults	The statistical result of every cell for the key metric defined at experiment setup. Empty when no result has been collected.

AdExperimentCellResult Schema

Field	Type	Description
cell	AdExperimentCell URN	The AdExperimentCell URN for which the result is generated. Of the form urn:li:adExperimentCell:(entity_urn,experiment_id) For example: urn:li:adExperimentCell:(urn:li:sponsoredCampaign:1234,567) for campaign ID 1234 in experiment ID 567.
confidenceInterval	ConfidenceInterval	The confidence interval of the AdExperimentKeyMetric for this cell, based on the confidence level specified at experiment setup.
confidenceInterval.lowerBound	double	The inclusive lower bound of the confidence interval.
confidenceInterval.upperBound	double	The inclusive upper bound of the confidence interval.

Sample Request

GET https://api.linkedin.com/rest/adExperimentResults/123

Sample Response

{
  "experimentId": 123,
  "generatedOn": {
    "day": 12,
    "month": 12,
    "year": 2018
  },
  "pValue": 0.05,
  "minimumDetectableEffect": 0.09,
  "winner": "urn:li:adExperimentCell:(urn:li:sponsoredCampaign:1234,123)",
  "cellResults": [
    {
      "cell": "urn:li:adExperimentCell:(urn:li:sponsoredCampaign:1234,123)",
      "confidenceInterval": {
        "upperBound": 5.62,
        "lowerBound": 4.39
      }
    },
    {
      "cell": "urn:li:adExperimentCell:(urn:li:sponsoredCampaign:5678,123)",
      "confidenceInterval": {
        "upperBound": 8.76,
        "lowerBound": 5.23
      }
    }
  ]
}

Exceptions

Following are expected exceptions from the Delete end point:

Code	HTTP Status	Message	Error Detail Type
FORBIDDEN	403	User has no access.
EXPERIMENT_NOT_FOUND	404	AdExperiment referenced in the request doesn't exist.
EXPERIMENT_RESULT_NOT_FOUND	404	AdExperiment referenced in the request exists. But corresponding result is not generated yet.

Find Experiments Results

Experiment results can be fetched in batch according to certain search criteria.

GET https://api.linkedin.com/rest/adExperimentResults?q=search&criteria=({criteria}:(values:List({value_list})))

Query Parameters

Parameter	Description	Format	Required
q	Designates the query finder. Must be search for the Experiment Finder.	String	Yes
searchByCriteria	Search criteria, the following enum values are supported: experimentIds	String	Yes

Search Criteria	Description	Value Format	Required
experimentIds	Search by list of experiment IDs.	List	No

Sample Request

The following sample request finds the experiment results for experiments with id 123 and 234.

GET https://api.linkedin.com/rest/adExperimentResults/?q=searchByCriteria&criteria=(experimentIds:List(123,234))

Sample Response

[
  {
    "winner": "urn:li:adExperimentCell:(5678,123)",
    "generatedOn": {
      "month": 3,
      "year": 2019,
      "day": 22
    },
    "cellResults": [
      {
        "confidenceInterval": {
          "upperBound": 2.5,
          "lowerBound": 1.2
        },
        "cell": "urn:li:adExperimentCell:(5678,123)"
      },
      {
        "confidenceInterval": {
          "upperBound": 2.5,
          "lowerBound": 1.2
        },
        "cell": "urn:li:adExperimentCell:(6789,123)"
      }
    ],
    "pValue": 0.05,
    "minimumDetectableEffect": 0.09,
    "experimentId": 123
  },
  {
    "winner": "urn:li:adExperimentCell:(7928534,234",
    "generatedOn": {
      "month": 6,
      "year": 2019,
      "day": 21
    },
    "cellResults": [
      {
        "confidenceInterval": {
          "upperBound": 3.5,
          "lowerBound": 2.2
        },
        "cell": "urn:li:adExperimentCell:(9015782,234)"
      },
      {
        "confidenceInterval": {
          "upperBound": 2.7,
          "lowerBound": 1.6
        },
        "cell": "urn:li:adExperimentCell:(7928534,234)"
      }
    ],
    "pValue": 0.06,
    "minimumDetectableEffect": 0.12,
    "experimentId": 234
  }
]

Exceptions

Finder requests should always succeed without any exception.

Error in Request Processing

Error response uses AdExperimentErrorDetail schema. It is similar to Validation Response but does not have level field. Most of the error code have been reused from validation response and few more values are allowed which are specific to exception.

AdExperimentErrorDetail Schema

An AdExperimentErrorDetail Schema is used to contain error information whenever there are multiple errors in any REST request. The schema of this is similar to AdExperimentValidationDetail schema with a difference that code contains some additional values that are not expected with validation of a single resource.

Field	Type	Description
details	Array of AdExperimentErrorDetailItem	An array of error detail item, each item representing one failure.

AdExperimentErrorDetailItem Schema

The AdExperimentErrorDetailItem schema is used to contain error information of a single error whenever there are multiple errors in any REST Request.

Field	Type	Description
code	AdExperimentErrorDetailCode	This is the code of failure.
message	string	A human-readable explanation of the error.
detail	AdExperimentValidationDetail	This is failure specific information associated with the experiment.

AdExperimentErrorDetailCode Schema

The AdExperimentErrorDetailCode shows all the errors that can be expected from the REST end point.

Code	Description
ACCOUNT_ALREADY_RUNNING_BRAND_LIFT_TEST	Account provided has a brand lift test with overlapping schedule. A/B test cannot run concurrently with a brand lift test.
ACCOUNT_NOT_FOUND	The ad account does not exist.
ACCOUNT_NOT_SERVABLE	System does not allow account to be served. For example, account is canceled. This cannot be reverted by user.
BATCH_GET_ENTITY_FROM_DIFFERENT_ACCOUNT	Batch Get request is trying to fetch entities from multiple accounts
CAMPAIGNS_FROM_DIFFERENT_ACCOUNT	The campaigns referenced in the experiment belong to two separate accounts.
CAMPAIGN_ALREADY_SCHEDULED	There is an already existing experiment with overlapping schedule containing same campaign.
CAMPAIGN_BID_MUST_BE_GREATER_THAN_MINIMUM	You must set a higher bid price than minimum for this campaign type.
CAMPAIGN_GROUPS_DIFFERENT	The campaigns in the experiment belong to different campaign groups under the same account.
CAMPAIGN_GROUP_DYNAMIC_BUDGET_OPTIMIZATION_MUST_BE_DISABLED	You should disable Campaign Group Dynamic Budget Optimization for valid A/B test.
CAMPAIGN_NOT_FOUND	The campaign does not exist.
CREATIVE_NOT_SERVABLE	System does not allow any creatives from one of the cells to be served. For example, all creatives are canceled in one campaign. This cannot be reverted by user.
DELETION_NOT_ALLOWED	Delete operation is not allowed except for experiments in PENDING execution state
DUPLICATE_ELEMENTS_IN_CELLS	Both cells reference the same entity.
EXPERIMENT_NOT_FOUND	AdExperiment referenced in the request doesn't exist
INVALID_CONFIDENCE_VALUE	Experiment’s confidence value must be >= 51 AND <= 99.
INVALID_END_TIME	Scheduled end time is not valid. It cannot be in the past or more than 180 days in the future.
INVALID_KEY_METRIC	The campaigns cannot provide information for the experiment's KeyMetric. For example, campaign ad format is not video, but experiment's KeyMetric is CPV.
INVALID_NO_OF_CELLS	The experiment must contain exactly 2 cells.
INVALID_SCHEDULE	Experiment’s schedule is not valid Experiment cannot be scheduled to run for more than 90 days.",
INVALID_START_TIME	Scheduled start time is not valid. It has to be positive.
INVALID_STATE_TRANSITION	The user is attempting to make a state change that is not allowed.
INVALID_SUM_OF_WEIGHT_OF_CELLS	The sum of traffic weight of cells must equal 100.
INVALID_WEIGHT_OF_CELL	A cell has an unexpected traffic weight. It must be >= 1, and <= 99.
SCHEMA_MISMATCH	Input does not match schema
SET_OR_UPDATE_NOT_ALLOWED	The request is attempting to either set or modify a field(s) which is not allowed.
WRITE_CONFLICT	User is trying to modify an entity which is already updated"

Sample Erroneous Request

POST https://api.linkedin.com/rest/adExperiments

{
  "name": "Experiment 1",
  "account": "urn:li:sponsoredAccount:123",
  "keyMetric": "CPC",
  "confidence": 90,
  "scheduledEndAt": 156211978530,
  "scheduledStartAt": -1561964400000,
  "userIntendedState": "ACTIVE",
  "autoStop": "AUTO_STOP_DISABLED",
  "cells": [
    {
      "name": "cell1",
      "trafficWeight": 50,
      "reference": {
        "campaign": "urn:li:sponsoredCampaign:123"
      }
    },
    {
      "name": "cell2",
      "trafficWeight": 50,
      "reference": {
        "campaign": "urn:li:sponsoredCampaign:345"
      }
    }
  ]
}

Sample Error Response

{
  "code": "MULTIPLE_VALIDATIONS_FAILED",
  "errorDetailType": "com.linkedin.ads.AdExperimentExceptionDetail",
  "errorDetails": {
    "details": [
      {
        "code": "INVALID_START_TIME",
        "detail": {
          "pathErrors": [
            {
              "input": {
                "inputPath": {
                  "fieldPath": "/scheduledStartAt"
                },
                "value": {
                  "decimalValue": "-156111971530"
                }
              }
            }
          ]
        },
        "message": "Scheduled start time is not valid."
      },
      {
        "code": "INVALID_END_TIME",
        "detail": {
          "pathErrors": [
            {
              "input": {
                "inputPath": {
                  "fieldPath": "/scheduledEndAt"
                },
                "value": {
                  "decimalValue": "156211978530"
                }
              }
            }
          ]
        },
        "message": "Scheduled end time is not valid."
      },
      {
        "code": "INVALID_SCHEDULE",
        "detail": {},
        "message": "Experiment's schedule is not valid. Experiment cannot be scheduled to run for more than 90 days."
      }
    ]
  },
  "exceptionClass": "com.linkedin.restli.server.RestLiServiceException",
  "status": 422
}

Experiment Cancellation or On Hold Reasons

An experiment can be stopped or put on hold at any point during its execution because of one or more of the following reasons:

Affected Entity	Action	Impact
Account	Deleted	Ad experiment would be marked as cancelled.
Campaign Group	Deleted	Ad experiment would be marked as cancelled.
	Paused	Ad experiment would be marked as cancelled.
	Total budget exists and it is exhausted	No impact on experiment.
	Daily budget is exhausted	No impact on experiment.
Campaign	Deleted	Ad experiment would be marked as cancelled.
	Paused	Ad experiment would be marked as cancelled.
	Total budget exists and it is exhausted	Ad experiment would be put on hold and cells would stop serving.
	Campaign is on hold because audience count is too low	Ad experiment would be put on hold and cells would stop serving.
	Campaigns end date is reached	Ad experiment would be put on hold and cells would stop serving. Would not result in experiment cancellation.
	Campaign start date is not reached yet	Ad experiment would be put on hold and cells would stop serving.
	Daily budget is exhausted	No impact on experiment.
Creative	Deleted	Ad experiment would be marked as cancelled if there are no more creatives under the same campaign which can serve.
	Paused	Ad experiment would be marked as cancelled.
	Rejected or Archived or Cancelled	Ad experiment would be marked as cancelled if there are no more creatives under the same campaign which can serve.
	In Review	Ad experiment would be put on hold and cells would stop serving.
Ad Form	Rejected or Cancelled	Ad experiment would be marked as cancelled if there are no more creatives under the same campaign which can serve.
	Deleted	Ad experiment would be marked as cancelled.
	In Review	Ad experiment would be put on hold and cells would stop serving.
Sponsored Content	Deleted	Ad experiment would be marked as cancelled.
	Purged by system because referred content in it is deleted	Ad experiment would be marked as cancelled.