Pipelines - Create Or Update

Creates or updates a pipeline.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines/{pipelineName}?api-version=2018-06-01

URI Parameters

Name In Required Type Description
factoryName
path True

string

The factory name.

Regex pattern: ^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*$

pipelineName
path True

string

The pipeline name.

Regex pattern: ^[A-Za-z0-9_][^<>*#.%&:\\+?/]*$

resourceGroupName
path True

string

The resource group name.

Regex pattern: ^[-\w\._\(\)]+$

subscriptionId
path True

string

The subscription identifier.

api-version
query True

string

The API version.

Request Header

Name Required Type Description
If-Match

string

ETag of the pipeline entity. Should only be specified for update, for which it should match existing entity or can be * for unconditional update.

Request Body

Name Type Description
properties.activities Activity[]:

List of activities in pipeline.

properties.annotations

object[]

List of tags that can be used for describing the Pipeline.

properties.concurrency

integer

The max number of concurrent runs for the pipeline.

properties.description

string

The description of the pipeline.

properties.folder

Folder

The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.

properties.parameters

<string,  ParameterSpecification>

List of parameters for pipeline.

properties.policy

PipelinePolicy

Pipeline Policy.

properties.runDimensions

object

Dimensions emitted by Pipeline.

properties.variables

<string,  VariableSpecification>

List of variables for pipeline.

Responses

Name Type Description
200 OK

PipelineResource

OK.

Other Status Codes

CloudError

An error response received from the Azure Data Factory service.

Security

azure_auth

Azure Active Directory OAuth2 Flow.

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name Description
user_impersonation impersonate your user account

Examples

Pipelines_Create
Pipelines_Update

Pipelines_Create

Sample request

PUT https://management.azure.com/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline?api-version=2018-06-01

{
  "properties": {
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      },
      "JobId": {
        "type": "String"
      }
    },
    "variables": {
      "TestVariableArray": {
        "type": "Array"
      }
    },
    "runDimensions": {
      "JobId": {
        "value": "@pipeline().parameters.JobId",
        "type": "Expression"
      }
    },
    "policy": {
      "elapsedTimeMetric": {
        "duration": "0.00:10:00"
      }
    }
  }
}

Sample response

Date: Sat, 16 Jun 2018 00:37:41 GMT
X-Content-Type-Options: nosniff
x-ms-ratelimit-remaining-subscription-writes: 1192
x-ms-request-id: e4c589b7-a9fe-4c28-981c-3855ec27d264
x-ms-correlation-request-id: e4c589b7-a9fe-4c28-981c-3855ec27d264
{
  "id": "/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline",
  "name": "examplePipeline",
  "type": "Microsoft.DataFactory/factories/pipelines",
  "properties": {
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      },
      "JobId": {
        "type": "String"
      }
    },
    "variables": {
      "TestVariableArray": {
        "type": "Array"
      }
    },
    "runDimensions": {
      "JobId": {
        "value": "@pipeline().parameters.JobId",
        "type": "Expression"
      }
    }
  },
  "etag": "0a0069d4-0000-0000-0000-5b245bd50000"
}

Pipelines_Update

Sample request

PUT https://management.azure.com/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline?api-version=2018-06-01

{
  "properties": {
    "description": "Example description",
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      }
    },
    "policy": {
      "elapsedTimeMetric": {
        "duration": "0.00:10:00"
      }
    }
  }
}

Sample response

Date: Sat, 16 Jun 2018 00:37:43 GMT
X-Content-Type-Options: nosniff
x-ms-ratelimit-remaining-subscription-writes: 1191
x-ms-request-id: 91a763b0-d944-493f-83fc-e989c8b6e122
x-ms-correlation-request-id: 91a763b0-d944-493f-83fc-e989c8b6e122
{
  "id": "/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline",
  "name": "examplePipeline",
  "type": "Microsoft.DataFactory/factories/pipelines",
  "properties": {
    "description": "Example description",
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      }
    }
  },
  "etag": "0a006cd4-0000-0000-0000-5b245bd60000"
}

Definitions

Name Description
ActivityDependency

Activity dependency information.

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

ActivityPolicy

Execution policy for an activity.

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

AppendVariableActivity

Append value for a Variable of type Array.

AzureDataExplorerCommandActivity

Azure Data Explorer command activity.

AzureFunctionActivity

Azure Function activity.

AzureFunctionActivityMethod

The list of HTTP methods supported by a AzureFunctionActivity.

AzureKeyVaultSecretReference

Azure Key Vault secret reference.

AzureMLBatchExecutionActivity

Azure ML Batch Execution activity.

AzureMLExecutePipelineActivity

Azure ML Execute Pipeline activity.

AzureMLUpdateResourceActivity

Azure ML Update Resource management activity.

AzureMLWebServiceFile

Azure ML WebService Input/Output file

BigDataPoolParametrizationReference

Big data pool reference type.

BigDataPoolReferenceType

Big data pool reference type.

CloudError

The object that defines the structure of an Azure Data Factory error response.

Compute

Compute properties for data flow activity.

ConfigurationType

The type of the spark config.

ContinuationSettingsReference

Continuation settings for execute data flow activity.

ControlActivity

Base class for all control activities like IfCondition, ForEach , Until.

CopyActivity

Copy activity.

CopyActivityLogSettings

Settings for copy activity log.

CredentialReference

Credential reference type.

CredentialReferenceType

Credential reference type.

CustomActivity

Custom activity type.

CustomActivityReferenceObject

Reference objects for custom activity

DatabricksNotebookActivity

DatabricksNotebook activity.

DatabricksSparkJarActivity

DatabricksSparkJar activity.

DatabricksSparkPythonActivity

DatabricksSparkPython activity.

DataFlowReference

Data flow reference type.

DataFlowReferenceType

Data flow reference type.

DataFlowStagingInfo

Staging info for execute data flow activity.

DataLakeAnalyticsUSQLActivity

Data Lake Analytics U-SQL activity.

DatasetReference

Dataset reference type.

DeleteActivity

Delete activity.

DependencyCondition

Match-Condition for the dependency.

ExecuteDataFlowActivity

Execute data flow activity.

ExecutePipelineActivity

Execute pipeline activity.

ExecutePipelineActivityPolicy

Execution policy for an execute pipeline activity.

ExecuteSSISPackageActivity

Execute SSIS package activity.

ExecuteWranglingDataflowActivity

Execute power query activity.

ExecutionActivity

Base class for all execution activities.

Expression

Azure Data Factory expression definition.

FailActivity

This activity will fail within its own scope and output a custom error message and error code. The error message and code can provided either as a string literal or as an expression that can be evaluated to a string at runtime. The activity scope can be the whole pipeline or a control activity (e.g. foreach, switch, until), if the fail activity is contained in it.

FilterActivity

Filter and return results from input array based on the conditions.

Folder

The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.

ForEachActivity

This activity is used for iterating over a collection and execute given activities.

GetMetadataActivity

Activity to get metadata of dataset

HDInsightActivityDebugInfoOption

The HDInsightActivityDebugInfoOption settings to use.

HDInsightHiveActivity

HDInsight Hive activity type.

HDInsightMapReduceActivity

HDInsight MapReduce activity type.

HDInsightPigActivity

HDInsight Pig activity type.

HDInsightSparkActivity

HDInsight Spark activity.

HDInsightStreamingActivity

HDInsight streaming activity type.

IfConditionActivity

This activity evaluates a boolean expression and executes either the activities under the ifTrueActivities property or the ifFalseActivities property depending on the result of the expression.

IntegrationRuntimeReference

Integration runtime reference type.

LinkedServiceReference

Linked service reference type.

LogLocationSettings

Log location settings.

LogSettings

Log settings.

LogStorageSettings

(Deprecated. Please use LogSettings) Log storage settings.

LookupActivity

Lookup activity.

NotebookParameter

Notebook parameter.

NotebookParameterType

Notebook parameter type.

NotebookReferenceType

Synapse notebook reference type.

ParameterSpecification

Definition of a single parameter for an entity.

ParameterType

Parameter type.

PipelineElapsedTimeMetricPolicy

Pipeline ElapsedTime Metric Policy.

PipelinePolicy

Pipeline Policy.

PipelineReference

Pipeline reference type.

PipelineResource

Pipeline resource type.

PowerQuerySink

Power query sink.

PowerQuerySinkMapping

Map Power Query mashup query to sink dataset(s).

RedirectIncompatibleRowSettings

Redirect incompatible row settings

ScriptActivity

Script activity type.

ScriptActivityLogDestination

The destination of logs. Type: string.

ScriptActivityParameter

Parameters of a script block.

ScriptActivityParameterDirection

The direction of the parameter.

ScriptActivityParameterType

The type of the parameter.

ScriptActivityScriptBlock

Script block of scripts.

ScriptActivityTypeProperties.LogSettings

Log settings of script activity.

SecureInputOutputPolicy

Execution policy for an activity that supports secure input and output.

SecureString

Azure Data Factory secure string definition. The string value will be masked with asterisks '*' during Get or List API calls.

SetVariableActivity

Set value for a Variable.

SkipErrorFile

Skip error file.

SparkConfigurationParametrizationReference

Spark configuration reference.

SparkConfigurationReferenceType

Spark configuration reference type.

SparkJobReferenceType

Synapse spark job reference type.

SqlServerStoredProcedureActivity

SQL stored procedure activity type.

SSISAccessCredential

SSIS access credential.

SSISChildPackage

SSIS embedded child package.

SSISExecutionCredential

SSIS package execution credential.

SSISExecutionParameter

SSIS execution parameter.

SSISLogLocation

SSIS package execution log location

SsisLogLocationType

The type of SSIS log location.

SSISPackageLocation

SSIS package location.

SsisPackageLocationType

The type of SSIS package location.

SSISPropertyOverride

SSIS property override.

StagingSettings

Staging settings.

SwitchActivity

This activity evaluates an expression and executes activities under the cases property that correspond to the expression evaluation expected in the equals property.

SwitchCase

Switch cases with have a value and corresponding activities.

SynapseNotebookActivity

Execute Synapse notebook activity.

SynapseNotebookReference

Synapse notebook reference type.

SynapseSparkJobDefinitionActivity

Execute spark job activity.

SynapseSparkJobReference

Synapse spark job reference type.

Type

Linked service reference type.

UntilActivity

This activity executes inner activities until the specified boolean expression results to true or timeout is reached, whichever is earlier.

UserProperty

User property.

ValidationActivity

This activity verifies that an external resource exists.

VariableSpecification

Definition of a single variable for a Pipeline.

VariableType

Variable type.

WaitActivity

This activity suspends pipeline execution for the specified interval.

WebActivity

Web activity.

WebActivityAuthentication

Web activity authentication properties.

WebActivityMethod

The list of HTTP methods supported by a WebActivity.

WebHookActivity

WebHook activity.

WebHookActivityMethod

The list of HTTP methods supported by a WebHook activity.

ActivityDependency

Activity dependency information.

Name Type Description
activity

string

Activity name.

dependencyConditions

DependencyCondition[]

Match-Condition for the dependency.

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

Name Type Description
Failed

string

Skipped

string

Succeeded

string

ActivityPolicy

Execution policy for an activity.

Name Type Description
retry

object

Maximum ordinary retry attempts. Default is 0. Type: integer (or Expression with resultType integer), minimum: 0.

retryIntervalInSeconds

integer

Interval between each retry attempt (in seconds). The default is 30 sec.

secureInput

boolean

When set to true, Input from activity is considered as secure and will not be logged to monitoring.

secureOutput

boolean

When set to true, Output from activity is considered as secure and will not be logged to monitoring.

timeout

object

Specifies the timeout for the activity to run. The default timeout is 7 days. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

Name Type Description
Active

string

Inactive

string

AppendVariableActivity

Append value for a Variable of type Array.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

AppendVariable

Type of activity.

typeProperties.value

object

Value to be appended. Type: could be a static value matching type of the variable item or Expression with resultType matching type of the variable item

typeProperties.variableName

string

Name of the variable whose value needs to be appended to.

userProperties

UserProperty[]

Activity user properties.

AzureDataExplorerCommandActivity

Azure Data Explorer command activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

AzureDataExplorerCommand

Type of activity.

typeProperties.command

object

A control command, according to the Azure Data Explorer command syntax. Type: string (or Expression with resultType string).

typeProperties.commandTimeout

object

Control command timeout. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9]))..)

userProperties

UserProperty[]

Activity user properties.

AzureFunctionActivity

Azure Function activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

AzureFunctionActivity

Type of activity.

typeProperties.body

object

Represents the payload that will be sent to the endpoint. Required for POST/PUT method, not allowed for GET method Type: string (or Expression with resultType string).

typeProperties.functionName

object

Name of the Function that the Azure Function Activity will call. Type: string (or Expression with resultType string)

typeProperties.headers

object

Represents the headers that will be sent to the request. For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. Type: string (or Expression with resultType string).

typeProperties.method

AzureFunctionActivityMethod

Rest API method for target endpoint.

userProperties

UserProperty[]

Activity user properties.

AzureFunctionActivityMethod

The list of HTTP methods supported by a AzureFunctionActivity.

Name Type Description
DELETE

string

GET

string

HEAD

string

OPTIONS

string

POST

string

PUT

string

TRACE

string

AzureKeyVaultSecretReference

Azure Key Vault secret reference.

Name Type Description
secretName

object

The name of the secret in Azure Key Vault. Type: string (or Expression with resultType string).

secretVersion

object

The version of the secret in Azure Key Vault. The default value is the latest version of the secret. Type: string (or Expression with resultType string).

store

LinkedServiceReference

The Azure Key Vault linked service reference.

type string:

AzureKeyVaultSecret

Type of the secret.

AzureMLBatchExecutionActivity

Azure ML Batch Execution activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

AzureMLBatchExecution

Type of activity.

typeProperties.globalParameters

object

Key,Value pairs to be passed to the Azure ML Batch Execution Service endpoint. Keys must match the names of web service parameters defined in the published Azure ML web service. Values will be passed in the GlobalParameters property of the Azure ML batch execution request.

typeProperties.webServiceInputs

<string,  AzureMLWebServiceFile>

Key,Value pairs, mapping the names of Azure ML endpoint's Web Service Inputs to AzureMLWebServiceFile objects specifying the input Blob locations.. This information will be passed in the WebServiceInputs property of the Azure ML batch execution request.

typeProperties.webServiceOutputs

<string,  AzureMLWebServiceFile>

Key,Value pairs, mapping the names of Azure ML endpoint's Web Service Outputs to AzureMLWebServiceFile objects specifying the output Blob locations. This information will be passed in the WebServiceOutputs property of the Azure ML batch execution request.

userProperties

UserProperty[]

Activity user properties.

AzureMLExecutePipelineActivity

Azure ML Execute Pipeline activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

AzureMLExecutePipeline

Type of activity.

typeProperties.continueOnStepFailure

object

Whether to continue execution of other steps in the PipelineRun if a step fails. This information will be passed in the continueOnStepFailure property of the published pipeline execution request. Type: boolean (or Expression with resultType boolean).

typeProperties.dataPathAssignments

object

Dictionary used for changing data path assignments without retraining. Values will be passed in the dataPathAssignments property of the published pipeline execution request. Type: object (or Expression with resultType object).

typeProperties.experimentName

object

Run history experiment name of the pipeline run. This information will be passed in the ExperimentName property of the published pipeline execution request. Type: string (or Expression with resultType string).

typeProperties.mlParentRunId

object

The parent Azure ML Service pipeline run id. This information will be passed in the ParentRunId property of the published pipeline execution request. Type: string (or Expression with resultType string).

typeProperties.mlPipelineEndpointId

object

ID of the published Azure ML pipeline endpoint. Type: string (or Expression with resultType string).

typeProperties.mlPipelineId

object

ID of the published Azure ML pipeline. Type: string (or Expression with resultType string).

typeProperties.mlPipelineParameters

object

Key,Value pairs to be passed to the published Azure ML pipeline endpoint. Keys must match the names of pipeline parameters defined in the published pipeline. Values will be passed in the ParameterAssignments property of the published pipeline execution request. Type: object with key value pairs (or Expression with resultType object).

typeProperties.version

object

Version of the published Azure ML pipeline endpoint. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

AzureMLUpdateResourceActivity

Azure ML Update Resource management activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

AzureMLUpdateResource

Type of activity.

typeProperties.trainedModelFilePath

object

The relative file path in trainedModelLinkedService to represent the .ilearner file that will be uploaded by the update operation. Type: string (or Expression with resultType string).

typeProperties.trainedModelLinkedServiceName

LinkedServiceReference

Name of Azure Storage linked service holding the .ilearner file that will be uploaded by the update operation.

typeProperties.trainedModelName

object

Name of the Trained Model module in the Web Service experiment to be updated. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

AzureMLWebServiceFile

Azure ML WebService Input/Output file

Name Type Description
filePath

object

The relative file path, including container name, in the Azure Blob Storage specified by the LinkedService. Type: string (or Expression with resultType string).

linkedServiceName

LinkedServiceReference

Reference to an Azure Storage LinkedService, where Azure ML WebService Input/Output file located.

BigDataPoolParametrizationReference

Big data pool reference type.

Name Type Description
referenceName

object

Reference big data pool name. Type: string (or Expression with resultType string).

type

BigDataPoolReferenceType

Big data pool reference type.

BigDataPoolReferenceType

Big data pool reference type.

Name Type Description
BigDataPoolReference

string

CloudError

The object that defines the structure of an Azure Data Factory error response.

Name Type Description
error.code

string

Error code.

error.details

CloudError[]

Array with additional error details.

error.message

string

Error message.

error.target

string

Property name/path in request associated with error.

Compute

Compute properties for data flow activity.

Name Type Description
computeType

object

Compute type of the cluster which will execute data flow job. Possible values include: 'General', 'MemoryOptimized', 'ComputeOptimized'. Type: string (or Expression with resultType string)

coreCount

object

Core count of the cluster which will execute data flow job. Supported values are: 8, 16, 32, 48, 80, 144 and 272. Type: integer (or Expression with resultType integer)

ConfigurationType

The type of the spark config.

Name Type Description
Artifact

string

Customized

string

Default

string

ContinuationSettingsReference

Continuation settings for execute data flow activity.

Name Type Description
continuationTtlInMinutes

object

Continuation TTL in minutes.

customizedCheckpointKey

object

Customized checkpoint key.

idleCondition

object

Idle condition.

ControlActivity

Base class for all control activities like IfCondition, ForEach , Until.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Container

Type of activity.

userProperties

UserProperty[]

Activity user properties.

CopyActivity

Copy activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

inputs

DatasetReference[]

List of inputs for the activity.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

outputs

DatasetReference[]

List of outputs for the activity.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Copy

Type of activity.

typeProperties.dataIntegrationUnits

object

Maximum number of data integration units that can be used to perform this data movement. Type: integer (or Expression with resultType integer), minimum: 0.

typeProperties.enableSkipIncompatibleRow

object

Whether to skip incompatible row. Default value is false. Type: boolean (or Expression with resultType boolean).

typeProperties.enableStaging

object

Specifies whether to copy data via an interim staging. Default value is false. Type: boolean (or Expression with resultType boolean).

typeProperties.logSettings

LogSettings

Log settings customer needs provide when enabling log.

typeProperties.logStorageSettings

LogStorageSettings

(Deprecated. Please use LogSettings) Log storage settings customer need to provide when enabling session log.

typeProperties.parallelCopies

object

Maximum number of concurrent sessions opened on the source or sink to avoid overloading the data store. Type: integer (or Expression with resultType integer), minimum: 0.

typeProperties.preserve

object[]

Preserve rules.

typeProperties.preserveRules

object[]

Preserve Rules.

typeProperties.redirectIncompatibleRowSettings

RedirectIncompatibleRowSettings

Redirect incompatible row settings when EnableSkipIncompatibleRow is true.

typeProperties.sink

CopySink

Copy activity sink.

typeProperties.skipErrorFile

SkipErrorFile

Specify the fault tolerance for data consistency.

typeProperties.source

CopySource

Copy activity source.

typeProperties.stagingSettings

StagingSettings

Specifies interim staging settings when EnableStaging is true.

typeProperties.translator

object

Copy activity translator. If not specified, tabular translator is used.

typeProperties.validateDataConsistency

object

Whether to enable Data Consistency validation. Type: boolean (or Expression with resultType boolean).

userProperties

UserProperty[]

Activity user properties.

CopyActivityLogSettings

Settings for copy activity log.

Name Type Description
enableReliableLogging

object

Specifies whether to enable reliable logging. Type: boolean (or Expression with resultType boolean).

logLevel

object

Gets or sets the log level, support: Info, Warning. Type: string (or Expression with resultType string).

CredentialReference

Credential reference type.

Name Type Description
referenceName

string

Reference credential name.

type

CredentialReferenceType

Credential reference type.

CredentialReferenceType

Credential reference type.

Name Type Description
CredentialReference

string

CustomActivity

Custom activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Custom

Type of activity.

typeProperties.autoUserSpecification

object

Elevation level and scope for the user, default is nonadmin task. Type: string (or Expression with resultType double).

typeProperties.command

object

Command for custom activity Type: string (or Expression with resultType string).

typeProperties.extendedProperties

object

User defined property bag. There is no restriction on the keys or values that can be used. The user specified custom activity has the full responsibility to consume and interpret the content defined.

typeProperties.folderPath

object

Folder path for resource files Type: string (or Expression with resultType string).

typeProperties.referenceObjects

CustomActivityReferenceObject

Reference objects

typeProperties.resourceLinkedService

LinkedServiceReference

Resource linked service reference.

typeProperties.retentionTimeInDays

object

The retention time for the files submitted for custom activity. Type: double (or Expression with resultType double).

userProperties

UserProperty[]

Activity user properties.

CustomActivityReferenceObject

Reference objects for custom activity

Name Type Description
datasets

DatasetReference[]

Dataset references.

linkedServices

LinkedServiceReference[]

Linked service references.

DatabricksNotebookActivity

DatabricksNotebook activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

DatabricksNotebook

Type of activity.

typeProperties.baseParameters

object

Base parameters to be used for each run of this job.If the notebook takes a parameter that is not specified, the default value from the notebook will be used.

typeProperties.libraries

object[]

A list of libraries to be installed on the cluster that will execute the job.

typeProperties.notebookPath

object

The absolute path of the notebook to be run in the Databricks Workspace. This path must begin with a slash. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

DatabricksSparkJarActivity

DatabricksSparkJar activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

DatabricksSparkJar

Type of activity.

typeProperties.libraries

object[]

A list of libraries to be installed on the cluster that will execute the job.

typeProperties.mainClassName

object

The full name of the class containing the main method to be executed. This class must be contained in a JAR provided as a library. Type: string (or Expression with resultType string).

typeProperties.parameters

object[]

Parameters that will be passed to the main method.

userProperties

UserProperty[]

Activity user properties.

DatabricksSparkPythonActivity

DatabricksSparkPython activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

DatabricksSparkPython

Type of activity.

typeProperties.libraries

object[]

A list of libraries to be installed on the cluster that will execute the job.

typeProperties.parameters

object[]

Command line parameters that will be passed to the Python file.

typeProperties.pythonFile

object

The URI of the Python file to be executed. DBFS paths are supported. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

DataFlowReference

Data flow reference type.

Name Type Description
datasetParameters

object

Reference data flow parameters from dataset.

parameters

object

Data flow parameters

referenceName

string

Reference data flow name.

type

DataFlowReferenceType

Data flow reference type.

DataFlowReferenceType

Data flow reference type.

Name Type Description
DataFlowReference

string

DataFlowStagingInfo

Staging info for execute data flow activity.

Name Type Description
folderPath

object

Folder path for staging blob. Type: string (or Expression with resultType string)

linkedService

LinkedServiceReference

Staging linked service reference.

DataLakeAnalyticsUSQLActivity

Data Lake Analytics U-SQL activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

DataLakeAnalyticsU-SQL

Type of activity.

typeProperties.compilationMode

object

Compilation mode of U-SQL. Must be one of these values : Semantic, Full and SingleBox. Type: string (or Expression with resultType string).

typeProperties.degreeOfParallelism

object

The maximum number of nodes simultaneously used to run the job. Default value is 1. Type: integer (or Expression with resultType integer), minimum: 1.

typeProperties.parameters

object

Parameters for U-SQL job request.

typeProperties.priority

object

Determines which jobs out of all that are queued should be selected to run first. The lower the number, the higher the priority. Default value is 1000. Type: integer (or Expression with resultType integer), minimum: 1.

typeProperties.runtimeVersion

object

Runtime version of the U-SQL engine to use. Type: string (or Expression with resultType string).

typeProperties.scriptLinkedService

LinkedServiceReference

Script linked service reference.

typeProperties.scriptPath

object

Case-sensitive path to folder that contains the U-SQL script. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

DatasetReference

Dataset reference type.

Name Type Description
parameters

object

Arguments for dataset.

referenceName

string

Reference dataset name.

type enum:

DatasetReference

Dataset reference type.

DeleteActivity

Delete activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Delete

Type of activity.

typeProperties.dataset

DatasetReference

Delete activity dataset reference.

typeProperties.enableLogging

object

Whether to record detailed logs of delete-activity execution. Default value is false. Type: boolean (or Expression with resultType boolean).

typeProperties.logStorageSettings

LogStorageSettings

Log storage settings customer need to provide when enableLogging is true.

typeProperties.maxConcurrentConnections

integer

The max concurrent connections to connect data source at the same time.

typeProperties.recursive

object

If true, files or sub-folders under current folder path will be deleted recursively. Default is false. Type: boolean (or Expression with resultType boolean).

typeProperties.storeSettings

StoreReadSettings

Delete activity store settings.

userProperties

UserProperty[]

Activity user properties.

DependencyCondition

Match-Condition for the dependency.

Name Type Description
Completed

string

Failed

string

Skipped

string

Succeeded

string

ExecuteDataFlowActivity

Execute data flow activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

ExecuteDataFlow

Type of activity.

typeProperties.compute

Compute

Compute properties for data flow activity.

typeProperties.continuationSettings

ContinuationSettingsReference

Continuation settings for execute data flow activity.

typeProperties.continueOnError

object

Continue on error setting used for data flow execution. Enables processing to continue if a sink fails. Type: boolean (or Expression with resultType boolean)

typeProperties.dataFlow

DataFlowReference

Data flow reference.

typeProperties.integrationRuntime

IntegrationRuntimeReference

The integration runtime reference.

typeProperties.runConcurrently

object

Concurrent run setting used for data flow execution. Allows sinks with the same save order to be processed concurrently. Type: boolean (or Expression with resultType boolean)

typeProperties.sourceStagingConcurrency

object

Specify number of parallel staging for sources applicable to the sink. Type: integer (or Expression with resultType integer)

typeProperties.staging

DataFlowStagingInfo

Staging info for execute data flow activity.

typeProperties.traceLevel

object

Trace level setting used for data flow monitoring output. Supported values are: 'coarse', 'fine', and 'none'. Type: string (or Expression with resultType string)

userProperties

UserProperty[]

Activity user properties.

ExecutePipelineActivity

Execute pipeline activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ExecutePipelineActivityPolicy

Execute pipeline activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

ExecutePipeline

Type of activity.

typeProperties.parameters

object

Pipeline parameters.

typeProperties.pipeline

PipelineReference

Pipeline reference.

typeProperties.waitOnCompletion

boolean

Defines whether activity execution will wait for the dependent pipeline execution to finish. Default is false.

userProperties

UserProperty[]

Activity user properties.

ExecutePipelineActivityPolicy

Execution policy for an execute pipeline activity.

Name Type Description
secureInput

boolean

When set to true, Input from activity is considered as secure and will not be logged to monitoring.

ExecuteSSISPackageActivity

Execute SSIS package activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

ExecuteSSISPackage

Type of activity.

typeProperties.connectVia

IntegrationRuntimeReference

The integration runtime reference.

typeProperties.environmentPath

object

The environment path to execute the SSIS package. Type: string (or Expression with resultType string).

typeProperties.executionCredential

SSISExecutionCredential

The package execution credential.

typeProperties.logLocation

SSISLogLocation

SSIS package execution log location.

typeProperties.loggingLevel

object

The logging level of SSIS package execution. Type: string (or Expression with resultType string).

typeProperties.packageConnectionManagers

object

The package level connection managers to execute the SSIS package.

typeProperties.packageLocation

SSISPackageLocation

SSIS package location.

typeProperties.packageParameters

<string,  SSISExecutionParameter>

The package level parameters to execute the SSIS package.

typeProperties.projectConnectionManagers

object

The project level connection managers to execute the SSIS package.

typeProperties.projectParameters

<string,  SSISExecutionParameter>

The project level parameters to execute the SSIS package.

typeProperties.propertyOverrides

<string,  SSISPropertyOverride>

The property overrides to execute the SSIS package.

typeProperties.runtime

object

Specifies the runtime to execute SSIS package. The value should be "x86" or "x64". Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

ExecuteWranglingDataflowActivity

Execute power query activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

ExecuteWranglingDataflow

Type of activity.

typeProperties.compute

Compute

Compute properties for data flow activity.

typeProperties.continuationSettings

ContinuationSettingsReference

Continuation settings for execute data flow activity.

typeProperties.continueOnError

object

Continue on error setting used for data flow execution. Enables processing to continue if a sink fails. Type: boolean (or Expression with resultType boolean)

typeProperties.dataFlow

DataFlowReference

Data flow reference.

typeProperties.integrationRuntime

IntegrationRuntimeReference

The integration runtime reference.

typeProperties.queries

PowerQuerySinkMapping[]

List of mapping for Power Query mashup query to sink dataset(s).

typeProperties.runConcurrently

object

Concurrent run setting used for data flow execution. Allows sinks with the same save order to be processed concurrently. Type: boolean (or Expression with resultType boolean)

typeProperties.sinks

<string,  PowerQuerySink>

(Deprecated. Please use Queries). List of Power Query activity sinks mapped to a queryName.

typeProperties.sourceStagingConcurrency

object

Specify number of parallel staging for sources applicable to the sink. Type: integer (or Expression with resultType integer)

typeProperties.staging

DataFlowStagingInfo

Staging info for execute data flow activity.

typeProperties.traceLevel

object

Trace level setting used for data flow monitoring output. Supported values are: 'coarse', 'fine', and 'none'. Type: string (or Expression with resultType string)

userProperties

UserProperty[]

Activity user properties.

ExecutionActivity

Base class for all execution activities.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Execution

Type of activity.

userProperties

UserProperty[]

Activity user properties.

Expression

Azure Data Factory expression definition.

Name Type Description
type enum:

Expression

Expression type.

value

string

Expression value.

FailActivity

This activity will fail within its own scope and output a custom error message and error code. The error message and code can provided either as a string literal or as an expression that can be evaluated to a string at runtime. The activity scope can be the whole pipeline or a control activity (e.g. foreach, switch, until), if the fail activity is contained in it.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Fail

Type of activity.

typeProperties.errorCode

object

The error code that categorizes the error type of the Fail activity. It can be dynamic content that's evaluated to a non empty/blank string at runtime. Type: string (or Expression with resultType string).

typeProperties.message

object

The error message that surfaced in the Fail activity. It can be dynamic content that's evaluated to a non empty/blank string at runtime. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

FilterActivity

Filter and return results from input array based on the conditions.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Filter

Type of activity.

typeProperties.condition

Expression

Condition to be used for filtering the input.

typeProperties.items

Expression

Input array on which filter should be applied.

userProperties

UserProperty[]

Activity user properties.

Folder

The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.

Name Type Description
name

string

The name of the folder that this Pipeline is in.

ForEachActivity

This activity is used for iterating over a collection and execute given activities.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

ForEach

Type of activity.

typeProperties.activities Activity[]:

List of activities to execute .

typeProperties.batchCount

integer

Batch count to be used for controlling the number of parallel execution (when isSequential is set to false).

typeProperties.isSequential

boolean

Should the loop be executed in sequence or in parallel (max 50)

typeProperties.items

Expression

Collection to iterate.

userProperties

UserProperty[]

Activity user properties.

GetMetadataActivity

Activity to get metadata of dataset

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

GetMetadata

Type of activity.

typeProperties.dataset

DatasetReference

GetMetadata activity dataset reference.

typeProperties.fieldList

object[]

Fields of metadata to get from dataset.

typeProperties.formatSettings

FormatReadSettings

GetMetadata activity format settings.

typeProperties.storeSettings

StoreReadSettings

GetMetadata activity store settings.

userProperties

UserProperty[]

Activity user properties.

HDInsightActivityDebugInfoOption

The HDInsightActivityDebugInfoOption settings to use.

Name Type Description
Always

string

Failure

string

None

string

HDInsightHiveActivity

HDInsight Hive activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

HDInsightHive

Type of activity.

typeProperties.arguments

object[]

User specified arguments to HDInsightActivity.

typeProperties.defines

object

Allows user to specify defines for Hive job request.

typeProperties.getDebugInfo

HDInsightActivityDebugInfoOption

Debug info option.

typeProperties.queryTimeout

integer

Query timeout value (in minutes). Effective when the HDInsight cluster is with ESP (Enterprise Security Package)

typeProperties.scriptLinkedService

LinkedServiceReference

Script linked service reference.

typeProperties.scriptPath

object

Script path. Type: string (or Expression with resultType string).

typeProperties.storageLinkedServices

LinkedServiceReference[]

Storage linked service references.

typeProperties.variables

object

User specified arguments under hivevar namespace.

userProperties

UserProperty[]

Activity user properties.

HDInsightMapReduceActivity

HDInsight MapReduce activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

HDInsightMapReduce

Type of activity.

typeProperties.arguments

object[]

User specified arguments to HDInsightActivity.

typeProperties.className

object

Class name. Type: string (or Expression with resultType string).

typeProperties.defines

object

Allows user to specify defines for the MapReduce job request.

typeProperties.getDebugInfo

HDInsightActivityDebugInfoOption

Debug info option.

typeProperties.jarFilePath

object

Jar path. Type: string (or Expression with resultType string).

typeProperties.jarLibs

object[]

Jar libs.

typeProperties.jarLinkedService

LinkedServiceReference

Jar linked service reference.

typeProperties.storageLinkedServices

LinkedServiceReference[]

Storage linked service references.

userProperties

UserProperty[]

Activity user properties.

HDInsightPigActivity

HDInsight Pig activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

HDInsightPig

Type of activity.

typeProperties.arguments

object

User specified arguments to HDInsightActivity. Type: array (or Expression with resultType array).

typeProperties.defines

object

Allows user to specify defines for Pig job request.

typeProperties.getDebugInfo

HDInsightActivityDebugInfoOption

Debug info option.

typeProperties.scriptLinkedService

LinkedServiceReference

Script linked service reference.

typeProperties.scriptPath

object

Script path. Type: string (or Expression with resultType string).

typeProperties.storageLinkedServices

LinkedServiceReference[]

Storage linked service references.

userProperties

UserProperty[]

Activity user properties.

HDInsightSparkActivity

HDInsight Spark activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

HDInsightSpark

Type of activity.

typeProperties.arguments

object[]

The user-specified arguments to HDInsightSparkActivity.

typeProperties.className

string

The application's Java/Spark main class.

typeProperties.entryFilePath

object

The relative path to the root folder of the code/package to be executed. Type: string (or Expression with resultType string).

typeProperties.getDebugInfo

HDInsightActivityDebugInfoOption

Debug info option.

typeProperties.proxyUser

object

The user to impersonate that will execute the job. Type: string (or Expression with resultType string).

typeProperties.rootPath

object

The root path in 'sparkJobLinkedService' for all the job’s files. Type: string (or Expression with resultType string).

typeProperties.sparkConfig

object

Spark configuration property.

typeProperties.sparkJobLinkedService

LinkedServiceReference

The storage linked service for uploading the entry file and dependencies, and for receiving logs.

userProperties

UserProperty[]

Activity user properties.

HDInsightStreamingActivity

HDInsight streaming activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

HDInsightStreaming

Type of activity.

typeProperties.arguments

object[]

User specified arguments to HDInsightActivity.

typeProperties.combiner

object

Combiner executable name. Type: string (or Expression with resultType string).

typeProperties.commandEnvironment

object[]

Command line environment values.

typeProperties.defines

object

Allows user to specify defines for streaming job request.

typeProperties.fileLinkedService

LinkedServiceReference

Linked service reference where the files are located.

typeProperties.filePaths

object[]

Paths to streaming job files. Can be directories.

typeProperties.getDebugInfo

HDInsightActivityDebugInfoOption

Debug info option.

typeProperties.input

object

Input blob path. Type: string (or Expression with resultType string).

typeProperties.mapper

object

Mapper executable name. Type: string (or Expression with resultType string).

typeProperties.output

object

Output blob path. Type: string (or Expression with resultType string).

typeProperties.reducer

object

Reducer executable name. Type: string (or Expression with resultType string).

typeProperties.storageLinkedServices

LinkedServiceReference[]

Storage linked service references.

userProperties

UserProperty[]

Activity user properties.

IfConditionActivity

This activity evaluates a boolean expression and executes either the activities under the ifTrueActivities property or the ifFalseActivities property depending on the result of the expression.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

IfCondition

Type of activity.

typeProperties.expression

Expression

An expression that would evaluate to Boolean. This is used to determine the block of activities (ifTrueActivities or ifFalseActivities) that will be executed.

typeProperties.ifFalseActivities Activity[]:

List of activities to execute if expression is evaluated to false. This is an optional property and if not provided, the activity will exit without any action.

typeProperties.ifTrueActivities Activity[]:

List of activities to execute if expression is evaluated to true. This is an optional property and if not provided, the activity will exit without any action.

userProperties

UserProperty[]

Activity user properties.

IntegrationRuntimeReference

Integration runtime reference type.

Name Type Description
parameters

object

Arguments for integration runtime.

referenceName

string

Reference integration runtime name.

type enum:

IntegrationRuntimeReference

Type of integration runtime.

LinkedServiceReference

Linked service reference type.

Name Type Description
parameters

object

Arguments for LinkedService.

referenceName

string

Reference LinkedService name.

type

Type

Linked service reference type.

LogLocationSettings

Log location settings.

Name Type Description
linkedServiceName

LinkedServiceReference

Log storage linked service reference.

path

object

The path to storage for storing detailed logs of activity execution. Type: string (or Expression with resultType string).

LogSettings

Log settings.

Name Type Description
copyActivityLogSettings

CopyActivityLogSettings

Specifies settings for copy activity log.

enableCopyActivityLog

object

Specifies whether to enable copy activity log. Type: boolean (or Expression with resultType boolean).

logLocationSettings

LogLocationSettings

Log location settings customer needs to provide when enabling log.

LogStorageSettings

(Deprecated. Please use LogSettings) Log storage settings.

Name Type Description
enableReliableLogging

object

Specifies whether to enable reliable logging. Type: boolean (or Expression with resultType boolean).

linkedServiceName

LinkedServiceReference

Log storage linked service reference.

logLevel

object

Gets or sets the log level, support: Info, Warning. Type: string (or Expression with resultType string).

path

object

The path to storage for storing detailed logs of activity execution. Type: string (or Expression with resultType string).

LookupActivity

Lookup activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Lookup

Type of activity.

typeProperties.dataset

DatasetReference

Lookup activity dataset reference.

typeProperties.firstRowOnly

object

Whether to return first row or all rows. Default value is true. Type: boolean (or Expression with resultType boolean).

typeProperties.source

CopySource

Dataset-specific source properties, same as copy activity source.

userProperties

UserProperty[]

Activity user properties.

NotebookParameter

Notebook parameter.

Name Type Description
type

NotebookParameterType

Notebook parameter type.

value

object

Notebook parameter value. Type: string (or Expression with resultType string).

NotebookParameterType

Notebook parameter type.

Name Type Description
bool

string

float

string

int

string

string

string

NotebookReferenceType

Synapse notebook reference type.

Name Type Description
NotebookReference

string

ParameterSpecification

Definition of a single parameter for an entity.

Name Type Description
defaultValue

object

Default value of parameter.

type

ParameterType

Parameter type.

ParameterType

Parameter type.

Name Type Description
Array

string

Bool

string

Float

string

Int

string

Object

string

SecureString

string

String

string

PipelineElapsedTimeMetricPolicy

Pipeline ElapsedTime Metric Policy.

Name Type Description
duration

object

TimeSpan value, after which an Azure Monitoring Metric is fired.

PipelinePolicy

Pipeline Policy.

Name Type Description
elapsedTimeMetric

PipelineElapsedTimeMetricPolicy

Pipeline ElapsedTime Metric Policy.

PipelineReference

Pipeline reference type.

Name Type Description
name

string

Reference name.

referenceName

string

Reference pipeline name.

type enum:

PipelineReference

Pipeline reference type.

PipelineResource

Pipeline resource type.

Name Type Description
etag

string

Etag identifies change in the resource.

id

string

The resource identifier.

name

string

The resource name.

properties.activities Activity[]:

List of activities in pipeline.

properties.annotations

object[]

List of tags that can be used for describing the Pipeline.

properties.concurrency

integer

The max number of concurrent runs for the pipeline.

properties.description

string

The description of the pipeline.

properties.folder

Folder

The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.

properties.parameters

<string,  ParameterSpecification>

List of parameters for pipeline.

properties.policy

PipelinePolicy

Pipeline Policy.

properties.runDimensions

object

Dimensions emitted by Pipeline.

properties.variables

<string,  VariableSpecification>

List of variables for pipeline.

type

string

The resource type.

PowerQuerySink

Power query sink.

Name Type Description
dataset

DatasetReference

Dataset reference.

description

string

Transformation description.

flowlet

DataFlowReference

Flowlet Reference

linkedService

LinkedServiceReference

Linked service reference.

name

string

Transformation name.

rejectedDataLinkedService

LinkedServiceReference

Rejected data linked service reference.

schemaLinkedService

LinkedServiceReference

Schema linked service reference.

script

string

sink script.

PowerQuerySinkMapping

Map Power Query mashup query to sink dataset(s).

Name Type Description
dataflowSinks

PowerQuerySink[]

List of sinks mapped to Power Query mashup query.

queryName

string

Name of the query in Power Query mashup document.

RedirectIncompatibleRowSettings

Redirect incompatible row settings

Name Type Description
linkedServiceName

object

Name of the Azure Storage, Storage SAS, or Azure Data Lake Store linked service used for redirecting incompatible row. Must be specified if redirectIncompatibleRowSettings is specified. Type: string (or Expression with resultType string).

path

object

The path for storing the redirect incompatible row data. Type: string (or Expression with resultType string).

ScriptActivity

Script activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Script

Type of activity.

typeProperties.logSettings

ScriptActivityTypeProperties.LogSettings

Log settings of script activity.

typeProperties.scriptBlockExecutionTimeout

object

ScriptBlock execution timeout. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

typeProperties.scripts

ScriptActivityScriptBlock[]

Array of script blocks. Type: array.

userProperties

UserProperty[]

Activity user properties.

ScriptActivityLogDestination

The destination of logs. Type: string.

Name Type Description
ActivityOutput

string

ExternalStore

string

ScriptActivityParameter

Parameters of a script block.

Name Type Description
direction

ScriptActivityParameterDirection

The direction of the parameter.

name

object

The name of the parameter. Type: string (or Expression with resultType string).

size

integer

The size of the output direction parameter.

type

ScriptActivityParameterType

The type of the parameter.

value

object

The value of the parameter. Type: string (or Expression with resultType string).

ScriptActivityParameterDirection

The direction of the parameter.

Name Type Description
Input

string

InputOutput

string

Output

string

ScriptActivityParameterType

The type of the parameter.

Name Type Description
Boolean

string

DateTime

string

DateTimeOffset

string

Decimal

string

Double

string

Guid

string

Int16

string

Int32

string

Int64

string

Single

string

String

string

Timespan

string

ScriptActivityScriptBlock

Script block of scripts.

Name Type Description
parameters

ScriptActivityParameter[]

Array of script parameters. Type: array.

text

object

The query text. Type: string (or Expression with resultType string).

type

object

The type of the query. Please refer to the ScriptType for valid options. Type: string (or Expression with resultType string).

ScriptActivityTypeProperties.LogSettings

Log settings of script activity.

Name Type Description
logDestination

ScriptActivityLogDestination

The destination of logs. Type: string.

logLocationSettings

LogLocationSettings

Log location settings customer needs to provide when enabling log.

SecureInputOutputPolicy

Execution policy for an activity that supports secure input and output.

Name Type Description
secureInput

boolean

When set to true, Input from activity is considered as secure and will not be logged to monitoring.

secureOutput

boolean

When set to true, Output from activity is considered as secure and will not be logged to monitoring.

SecureString

Azure Data Factory secure string definition. The string value will be masked with asterisks '*' during Get or List API calls.

Name Type Description
type string:

SecureString

Type of the secret.

value

string

Value of secure string.

SetVariableActivity

Set value for a Variable.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

SecureInputOutputPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

SetVariable

Type of activity.

typeProperties.setSystemVariable

boolean

If set to true, it sets the pipeline run return value.

typeProperties.value

object

Value to be set. Could be a static value or Expression.

typeProperties.variableName

string

Name of the variable whose value needs to be set.

userProperties

UserProperty[]

Activity user properties.

SkipErrorFile

Skip error file.

Name Type Description
dataInconsistency

object

Skip if source/sink file changed by other concurrent write. Default is false. Type: boolean (or Expression with resultType boolean).

fileMissing

object

Skip if file is deleted by other client during copy. Default is true. Type: boolean (or Expression with resultType boolean).

SparkConfigurationParametrizationReference

Spark configuration reference.

Name Type Description
referenceName

object

Reference spark configuration name. Type: string (or Expression with resultType string).

type

SparkConfigurationReferenceType

Spark configuration reference type.

SparkConfigurationReferenceType

Spark configuration reference type.

Name Type Description
SparkConfigurationReference

string

SparkJobReferenceType

Synapse spark job reference type.

Name Type Description
SparkJobDefinitionReference

string

SqlServerStoredProcedureActivity

SQL stored procedure activity type.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

SqlServerStoredProcedure

Type of activity.

typeProperties.storedProcedureName

object

Stored procedure name. Type: string (or Expression with resultType string).

typeProperties.storedProcedureParameters

object

Value and type setting for stored procedure parameters. Example: "{Parameter1: {value: "1", type: "int"}}".

userProperties

UserProperty[]

Activity user properties.

SSISAccessCredential

SSIS access credential.

Name Type Description
domain

object

Domain for windows authentication. Type: string (or Expression with resultType string).

password SecretBase:

Password for windows authentication.

userName

object

UseName for windows authentication. Type: string (or Expression with resultType string).

SSISChildPackage

SSIS embedded child package.

Name Type Description
packageContent

object

Content for embedded child package. Type: string (or Expression with resultType string).

packageLastModifiedDate

string

Last modified date for embedded child package.

packageName

string

Name for embedded child package.

packagePath

object

Path for embedded child package. Type: string (or Expression with resultType string).

SSISExecutionCredential

SSIS package execution credential.

Name Type Description
domain

object

Domain for windows authentication. Type: string (or Expression with resultType string).

password

SecureString

Password for windows authentication.

userName

object

UseName for windows authentication. Type: string (or Expression with resultType string).

SSISExecutionParameter

SSIS execution parameter.

Name Type Description
value

object

SSIS package execution parameter value. Type: string (or Expression with resultType string).

SSISLogLocation

SSIS package execution log location

Name Type Description
logPath

object

The SSIS package execution log path. Type: string (or Expression with resultType string).

type

SsisLogLocationType

The type of SSIS log location.

typeProperties.accessCredential

SSISAccessCredential

The package execution log access credential.

typeProperties.logRefreshInterval

object

Specifies the interval to refresh log. The default interval is 5 minutes. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

SsisLogLocationType

The type of SSIS log location.

Name Type Description
File

string

SSISPackageLocation

SSIS package location.

Name Type Description
packagePath

object

The SSIS package path. Type: string (or Expression with resultType string).

type

SsisPackageLocationType

The type of SSIS package location.

typeProperties.accessCredential

SSISAccessCredential

The package access credential.

typeProperties.childPackages

SSISChildPackage[]

The embedded child package list.

typeProperties.configurationAccessCredential

SSISAccessCredential

The configuration file access credential.

typeProperties.configurationPath

object

The configuration file of the package execution. Type: string (or Expression with resultType string).

typeProperties.packageContent

object

The embedded package content. Type: string (or Expression with resultType string).

typeProperties.packageLastModifiedDate

string

The embedded package last modified date.

typeProperties.packageName

string

The package name.

typeProperties.packagePassword SecretBase:

Password of the package.

SsisPackageLocationType

The type of SSIS package location.

Name Type Description
File

string

InlinePackage

string

PackageStore

string

SSISDB

string

SSISPropertyOverride

SSIS property override.

Name Type Description
isSensitive

boolean

Whether SSIS package property override value is sensitive data. Value will be encrypted in SSISDB if it is true

value

object

SSIS package property override value. Type: string (or Expression with resultType string).

StagingSettings

Staging settings.

Name Type Description
enableCompression

object

Specifies whether to use compression when copying data via an interim staging. Default value is false. Type: boolean (or Expression with resultType boolean).

linkedServiceName

LinkedServiceReference

Staging linked service reference.

path

object

The path to storage for storing the interim data. Type: string (or Expression with resultType string).

SwitchActivity

This activity evaluates an expression and executes activities under the cases property that correspond to the expression evaluation expected in the equals property.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Switch

Type of activity.

typeProperties.cases

SwitchCase[]

List of cases that correspond to expected values of the 'on' property. This is an optional property and if not provided, the activity will execute activities provided in defaultActivities.

typeProperties.defaultActivities Activity[]:

List of activities to execute if no case condition is satisfied. This is an optional property and if not provided, the activity will exit without any action.

typeProperties.on

Expression

An expression that would evaluate to a string or integer. This is used to determine the block of activities in cases that will be executed.

userProperties

UserProperty[]

Activity user properties.

SwitchCase

Switch cases with have a value and corresponding activities.

Name Type Description
activities Activity[]:

List of activities to execute for satisfied case condition.

value

string

Expected value that satisfies the expression result of the 'on' property.

SynapseNotebookActivity

Execute Synapse notebook activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

SynapseNotebook

Type of activity.

typeProperties.conf

object

Spark configuration properties, which will override the 'conf' of the notebook you provide.

typeProperties.configurationType

ConfigurationType

The type of the spark config.

typeProperties.driverSize

object

Number of core and memory to be used for driver allocated in the specified Spark pool for the session, which will be used for overriding 'driverCores' and 'driverMemory' of the notebook you provide. Type: string (or Expression with resultType string).

typeProperties.executorSize

object

Number of core and memory to be used for executors allocated in the specified Spark pool for the session, which will be used for overriding 'executorCores' and 'executorMemory' of the notebook you provide. Type: string (or Expression with resultType string).

typeProperties.notebook

SynapseNotebookReference

Synapse notebook reference.

typeProperties.numExecutors

object

Number of executors to launch for this session, which will override the 'numExecutors' of the notebook you provide. Type: integer (or Expression with resultType integer).

typeProperties.parameters

<string,  NotebookParameter>

Notebook parameters.

typeProperties.sparkConfig

object

Spark configuration property.

typeProperties.sparkPool

BigDataPoolParametrizationReference

The name of the big data pool which will be used to execute the notebook.

typeProperties.targetSparkConfiguration

SparkConfigurationParametrizationReference

The spark configuration of the spark job.

userProperties

UserProperty[]

Activity user properties.

SynapseNotebookReference

Synapse notebook reference type.

Name Type Description
referenceName

object

Reference notebook name. Type: string (or Expression with resultType string).

type

NotebookReferenceType

Synapse notebook reference type.

SynapseSparkJobDefinitionActivity

Execute spark job activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

SparkJob

Type of activity.

typeProperties.args

SynapseSparkJobActivityTypeProperties.Args[]

User specified arguments to SynapseSparkJobDefinitionActivity.

typeProperties.className

object

The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).

typeProperties.conf

object

Spark configuration properties, which will override the 'conf' of the spark job definition you provide.

typeProperties.configurationType

ConfigurationType

The type of the spark config.

typeProperties.driverSize

object

Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

typeProperties.executorSize

object

Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).

typeProperties.file

object

The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).

typeProperties.files

object[]

(Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.

typeProperties.filesV2

object[]

Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.

typeProperties.numExecutors

object

Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).

typeProperties.pythonCodeReference

object[]

Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.

typeProperties.scanFolder

object

Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).

typeProperties.sparkConfig

object

Spark configuration property.

typeProperties.sparkJob

SynapseSparkJobReference

Synapse spark job reference.

typeProperties.targetBigDataPool

BigDataPoolParametrizationReference

The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.

typeProperties.targetSparkConfiguration

SparkConfigurationParametrizationReference

The spark configuration of the spark job.

userProperties

UserProperty[]

Activity user properties.

SynapseSparkJobReference

Synapse spark job reference type.

Name Type Description
referenceName

object

Reference spark job name. Expression with resultType string.

type

SparkJobReferenceType

Synapse spark job reference type.

Type

Linked service reference type.

Name Type Description
LinkedServiceReference

string

UntilActivity

This activity executes inner activities until the specified boolean expression results to true or timeout is reached, whichever is earlier.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Until

Type of activity.

typeProperties.activities Activity[]:

List of activities to execute.

typeProperties.expression

Expression

An expression that would evaluate to Boolean. The loop will continue until this expression evaluates to true

typeProperties.timeout

object

Specifies the timeout for the activity to run. If there is no value specified, it takes the value of TimeSpan.FromDays(7) which is 1 week as default. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

userProperties

UserProperty[]

Activity user properties.

UserProperty

User property.

Name Type Description
name

string

User property name.

value

object

User property value. Type: string (or Expression with resultType string).

ValidationActivity

This activity verifies that an external resource exists.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Validation

Type of activity.

typeProperties.childItems

object

Can be used if dataset points to a folder. If set to true, the folder must have at least one file. If set to false, the folder must be empty. Type: boolean (or Expression with resultType boolean).

typeProperties.dataset

DatasetReference

Validation activity dataset reference.

typeProperties.minimumSize

object

Can be used if dataset points to a file. The file must be greater than or equal in size to the value specified. Type: integer (or Expression with resultType integer).

typeProperties.sleep

object

A delay in seconds between validation attempts. If no value is specified, 10 seconds will be used as the default. Type: integer (or Expression with resultType integer).

typeProperties.timeout

object

Specifies the timeout for the activity to run. If there is no value specified, it takes the value of TimeSpan.FromDays(7) which is 1 week as default. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

userProperties

UserProperty[]

Activity user properties.

VariableSpecification

Definition of a single variable for a Pipeline.

Name Type Description
defaultValue

object

Default value of variable.

type

VariableType

Variable type.

VariableType

Variable type.

Name Type Description
Array

string

Bool

string

String

string

WaitActivity

This activity suspends pipeline execution for the specified interval.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

Wait

Type of activity.

typeProperties.waitTimeInSeconds

object

Duration in seconds. Type: integer (or Expression with resultType integer).

userProperties

UserProperty[]

Activity user properties.

WebActivity

Web activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

linkedServiceName

LinkedServiceReference

Linked service reference.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

ActivityPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

WebActivity

Type of activity.

typeProperties.authentication

WebActivityAuthentication

Authentication method used for calling the endpoint.

typeProperties.body

object

Represents the payload that will be sent to the endpoint. Required for POST/PUT method, not allowed for GET method Type: string (or Expression with resultType string).

typeProperties.connectVia

IntegrationRuntimeReference

The integration runtime reference.

typeProperties.datasets

DatasetReference[]

List of datasets passed to web endpoint.

typeProperties.disableCertValidation

boolean

When set to true, Certificate validation will be disabled.

typeProperties.headers

object

Represents the headers that will be sent to the request. For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. Type: string (or Expression with resultType string).

typeProperties.httpRequestTimeout

object

Timeout for the HTTP request to get a response. Format is in TimeSpan (hh:mm:ss). This value is the timeout to get a response, not the activity timeout. The default value is 00:01:00 (1 minute). The range is from 1 to 10 minutes

typeProperties.linkedServices

LinkedServiceReference[]

List of linked services passed to web endpoint.

typeProperties.method

WebActivityMethod

Rest API method for target endpoint.

typeProperties.turnOffAsync

boolean

Option to disable invoking HTTP GET on location given in response header of a HTTP 202 Response. If set true, it stops invoking HTTP GET on http location given in response header. If set false then continues to invoke HTTP GET call on location given in http response headers.

typeProperties.url

object

Web activity target endpoint and path. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

WebActivityAuthentication

Web activity authentication properties.

Name Type Description
credential

CredentialReference

The credential reference containing authentication information.

password SecretBase:

Password for the PFX file or basic authentication / Secret when used for ServicePrincipal

pfx SecretBase:

Base64-encoded contents of a PFX file or Certificate when used for ServicePrincipal

resource

object

Resource for which Azure Auth token will be requested when using MSI Authentication. Type: string (or Expression with resultType string).

type

string

Web activity authentication (Basic/ClientCertificate/MSI/ServicePrincipal)

userTenant

object

TenantId for which Azure Auth token will be requested when using ServicePrincipal Authentication. Type: string (or Expression with resultType string).

username

object

Web activity authentication user name for basic authentication or ClientID when used for ServicePrincipal. Type: string (or Expression with resultType string).

WebActivityMethod

The list of HTTP methods supported by a WebActivity.

Name Type Description
DELETE

string

GET

string

POST

string

PUT

string

WebHookActivity

WebHook activity.

Name Type Description
dependsOn

ActivityDependency[]

Activity depends on condition.

description

string

Activity description.

name

string

Activity name.

onInactiveMarkAs

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

policy

SecureInputOutputPolicy

Activity policy.

state

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

type string:

WebHook

Type of activity.

typeProperties.authentication

WebActivityAuthentication

Authentication method used for calling the endpoint.

typeProperties.body

object

Represents the payload that will be sent to the endpoint. Required for POST/PUT method, not allowed for GET method Type: string (or Expression with resultType string).

typeProperties.headers

object

Represents the headers that will be sent to the request. For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. Type: string (or Expression with resultType string).

typeProperties.method

WebHookActivityMethod

Rest API method for target endpoint.

typeProperties.reportStatusOnCallBack

object

When set to true, statusCode, output and error in callback request body will be consumed by activity. The activity can be marked as failed by setting statusCode >= 400 in callback request. Default is false. Type: boolean (or Expression with resultType boolean).

typeProperties.timeout

string

The timeout within which the webhook should be called back. If there is no value specified, it defaults to 10 minutes. Type: string. Pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

typeProperties.url

object

WebHook activity target endpoint and path. Type: string (or Expression with resultType string).

userProperties

UserProperty[]

Activity user properties.

WebHookActivityMethod

The list of HTTP methods supported by a WebHook activity.

Name Type Description
POST

string