Pipelines - Create Or Update

Service:

Data Factory

API Version:

2018-06-01

Creates or updates a pipeline.

PUT https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/pipelines/{pipelineName}?api-version=2018-06-01

URI Parameters

Name	In	Required	Type	Description
factoryName	path	True	string	The factory name. Regex pattern: ^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*$
pipelineName	path	True	string	The pipeline name. Regex pattern: ^[A-Za-z0-9_][^<>*#.%&:\\+?/]*$
resourceGroupName	path	True	string	The resource group name. Regex pattern: ^[-\w\._\(\)]+$
subscriptionId	path	True	string	The subscription identifier.
api-version	query	True	string	The API version.

Request Header

Name	Required	Type	Description
If-Match		string	ETag of the pipeline entity. Should only be specified for update, for which it should match existing entity or can be * for unconditional update.

Request Body

Name	Type	Description
properties.activities	Activity[]: ControlActivity[] ExecutionActivity[] ExecuteWranglingDataflowActivity[] CopyActivity[] HDInsightHiveActivity[] HDInsightPigActivity[] HDInsightMapReduceActivity[] HDInsightStreamingActivity[] HDInsightSparkActivity[] ExecuteSSISPackageActivity[] CustomActivity[] SqlServerStoredProcedureActivity[] DeleteActivity[] AzureDataExplorerCommandActivity[] LookupActivity[] WebActivity[] GetMetadataActivity[] AzureMLBatchExecutionActivity[] AzureMLUpdateResourceActivity[] AzureMLExecutePipelineActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] AzureFunctionActivity[] ExecuteDataFlowActivity[] ScriptActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] ExecutePipelineActivity[] IfConditionActivity[] SwitchActivity[] ForEachActivity[] WaitActivity[] FailActivity[] UntilActivity[] ValidationActivity[] FilterActivity[] SetVariableActivity[] AppendVariableActivity[] WebHookActivity[]	List of activities in pipeline.
properties.annotations	object[]	List of tags that can be used for describing the Pipeline.
properties.concurrency	integer	The max number of concurrent runs for the pipeline.
properties.description	string	The description of the pipeline.
properties.folder	Folder	The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.
properties.parameters	<string,  ParameterSpecification>	List of parameters for pipeline.
properties.policy	PipelinePolicy	Pipeline Policy.
properties.runDimensions	object	Dimensions emitted by Pipeline.
properties.variables	<string,  VariableSpecification>	List of variables for pipeline.

Responses

Name	Type	Description
200 OK	PipelineResource	OK.
Other Status Codes	CloudError	An error response received from the Azure Data Factory service.

Security

azure_auth

Azure Active Directory OAuth2 Flow.

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name	Description
user_impersonation	impersonate your user account

Examples

Pipelines_Create

Pipelines_Update

Pipelines_Create

Sample Request

HTTP

PUT https://management.azure.com/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline?api-version=2018-06-01

{
  "properties": {
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      },
      "JobId": {
        "type": "String"
      }
    },
    "variables": {
      "TestVariableArray": {
        "type": "Array"
      }
    },
    "runDimensions": {
      "JobId": {
        "value": "@pipeline().parameters.JobId",
        "type": "Expression"
      }
    },
    "policy": {
      "elapsedTimeMetric": {
        "duration": "0.00:10:00"
      }
    }
  }
}

Java

import com.azure.core.management.serializer.SerializerFactory;
import com.azure.core.util.serializer.SerializerEncoding;
import com.azure.resourcemanager.datafactory.models.BlobSink;
import com.azure.resourcemanager.datafactory.models.BlobSource;
import com.azure.resourcemanager.datafactory.models.CopyActivity;
import com.azure.resourcemanager.datafactory.models.DatasetReference;
import com.azure.resourcemanager.datafactory.models.Expression;
import com.azure.resourcemanager.datafactory.models.ForEachActivity;
import com.azure.resourcemanager.datafactory.models.ParameterSpecification;
import com.azure.resourcemanager.datafactory.models.ParameterType;
import com.azure.resourcemanager.datafactory.models.PipelineElapsedTimeMetricPolicy;
import com.azure.resourcemanager.datafactory.models.PipelinePolicy;
import com.azure.resourcemanager.datafactory.models.VariableSpecification;
import com.azure.resourcemanager.datafactory.models.VariableType;
import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

/**
 * Samples for Pipelines CreateOrUpdate.
 */
public final class Main {
    /*
     * x-ms-original-file:
     * specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Create.json
     */
    /**
     * Sample code: Pipelines_Create.
     * 
     * @param manager Entry point to DataFactoryManager.
     */
    public static void pipelinesCreate(com.azure.resourcemanager.datafactory.DataFactoryManager manager)
        throws IOException {
        manager.pipelines().define("examplePipeline").withExistingFactory("exampleResourceGroup", "exampleFactoryName")
            .withActivities(Arrays.asList(new ForEachActivity().withName("ExampleForeachActivity")
                .withIsSequential(true)
                .withItems(new Expression().withValue("@pipeline().parameters.OutputBlobNameList"))
                .withActivities(Arrays.asList(new CopyActivity().withName("ExampleCopyActivity")
                    .withInputs(Arrays.asList(new DatasetReference().withReferenceName("exampleDataset").withParameters(
                        mapOf("MyFileName", "examplecontainer.csv", "MyFolderPath", "examplecontainer"))))
                    .withOutputs(Arrays.asList(new DatasetReference().withReferenceName("exampleDataset")
                        .withParameters(mapOf("MyFileName",
                            SerializerFactory.createDefaultManagementSerializerAdapter().deserialize(
                                "{\"type\":\"Expression\",\"value\":\"@item()\"}", Object.class,
                                SerializerEncoding.JSON),
                            "MyFolderPath", "examplecontainer"))))
                    .withSource(new BlobSource()).withSink(new BlobSink()).withDataIntegrationUnits(32)))))
            .withParameters(mapOf("JobId", new ParameterSpecification().withType(ParameterType.STRING),
                "OutputBlobNameList", new ParameterSpecification().withType(ParameterType.ARRAY)))
            .withVariables(mapOf("TestVariableArray", new VariableSpecification().withType(VariableType.ARRAY)))
            .withRunDimensions(mapOf("JobId",
                SerializerFactory.createDefaultManagementSerializerAdapter().deserialize(
                    "{\"type\":\"Expression\",\"value\":\"@pipeline().parameters.JobId\"}", Object.class,
                    SerializerEncoding.JSON)))
            .withPolicy(new PipelinePolicy()
                .withElapsedTimeMetric(new PipelineElapsedTimeMetricPolicy().withDuration("0.00:10:00")))
            .create();
    }

    // Use "Map.of" if available
    @SuppressWarnings("unchecked")
    private static <T> Map<String, T> mapOf(Object... inputs) {
        Map<String, T> map = new HashMap<>();
        for (int i = 0; i < inputs.length; i += 2) {
            String key = (String) inputs[i];
            T value = (T) inputs[i + 1];
            map.put(key, value);
        }
        return map;
    }
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Python

from azure.identity import DefaultAzureCredential
from azure.mgmt.datafactory import DataFactoryManagementClient

"""
# PREREQUISITES
    pip install azure-identity
    pip install azure-mgmt-datafactory
# USAGE
    python pipelines_create.py

    Before run the sample, please set the values of the client ID, tenant ID and client secret
    of the AAD application as environment variables: AZURE_CLIENT_ID, AZURE_TENANT_ID,
    AZURE_CLIENT_SECRET. For more info about how to get the value, please see:
    https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal
"""


def main():
    client = DataFactoryManagementClient(
        credential=DefaultAzureCredential(),
        subscription_id="12345678-1234-1234-1234-12345678abc",
    )

    response = client.pipelines.create_or_update(
        resource_group_name="exampleResourceGroup",
        factory_name="exampleFactoryName",
        pipeline_name="examplePipeline",
        pipeline={
            "properties": {
                "activities": [
                    {
                        "name": "ExampleForeachActivity",
                        "type": "ForEach",
                        "typeProperties": {
                            "activities": [
                                {
                                    "inputs": [
                                        {
                                            "parameters": {
                                                "MyFileName": "examplecontainer.csv",
                                                "MyFolderPath": "examplecontainer",
                                            },
                                            "referenceName": "exampleDataset",
                                            "type": "DatasetReference",
                                        }
                                    ],
                                    "name": "ExampleCopyActivity",
                                    "outputs": [
                                        {
                                            "parameters": {
                                                "MyFileName": {"type": "Expression", "value": "@item()"},
                                                "MyFolderPath": "examplecontainer",
                                            },
                                            "referenceName": "exampleDataset",
                                            "type": "DatasetReference",
                                        }
                                    ],
                                    "type": "Copy",
                                    "typeProperties": {
                                        "dataIntegrationUnits": 32,
                                        "sink": {"type": "BlobSink"},
                                        "source": {"type": "BlobSource"},
                                    },
                                }
                            ],
                            "isSequential": True,
                            "items": {"type": "Expression", "value": "@pipeline().parameters.OutputBlobNameList"},
                        },
                    }
                ],
                "parameters": {"JobId": {"type": "String"}, "OutputBlobNameList": {"type": "Array"}},
                "policy": {"elapsedTimeMetric": {"duration": "0.00:10:00"}},
                "runDimensions": {"JobId": {"type": "Expression", "value": "@pipeline().parameters.JobId"}},
                "variables": {"TestVariableArray": {"type": "Array"}},
            }
        },
    )
    print(response)


# x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Create.json
if __name__ == "__main__":
    main()

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Go

package armdatafactory_test

import (
	"context"
	"log"

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/datafactory/armdatafactory/v6"
)

// Generated from example definition: https://github.com/Azure/azure-rest-api-specs/blob/01a71545e82bb98b8137d3038150c436d46a59ed/specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Create.json
func ExamplePipelinesClient_CreateOrUpdate_pipelinesCreate() {
	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		log.Fatalf("failed to obtain a credential: %v", err)
	}
	ctx := context.Background()
	clientFactory, err := armdatafactory.NewClientFactory("<subscription-id>", cred, nil)
	if err != nil {
		log.Fatalf("failed to create client: %v", err)
	}
	res, err := clientFactory.NewPipelinesClient().CreateOrUpdate(ctx, "exampleResourceGroup", "exampleFactoryName", "examplePipeline", armdatafactory.PipelineResource{
		Properties: &armdatafactory.Pipeline{
			Activities: []armdatafactory.ActivityClassification{
				&armdatafactory.ForEachActivity{
					Name: to.Ptr("ExampleForeachActivity"),
					Type: to.Ptr("ForEach"),
					TypeProperties: &armdatafactory.ForEachActivityTypeProperties{
						Activities: []armdatafactory.ActivityClassification{
							&armdatafactory.CopyActivity{
								Name: to.Ptr("ExampleCopyActivity"),
								Type: to.Ptr("Copy"),
								Inputs: []*armdatafactory.DatasetReference{
									{
										Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
										Parameters: map[string]any{
											"MyFileName":   "examplecontainer.csv",
											"MyFolderPath": "examplecontainer",
										},
										ReferenceName: to.Ptr("exampleDataset"),
									}},
								Outputs: []*armdatafactory.DatasetReference{
									{
										Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
										Parameters: map[string]any{
											"MyFileName": map[string]any{
												"type":  "Expression",
												"value": "@item()",
											},
											"MyFolderPath": "examplecontainer",
										},
										ReferenceName: to.Ptr("exampleDataset"),
									}},
								TypeProperties: &armdatafactory.CopyActivityTypeProperties{
									DataIntegrationUnits: float64(32),
									Sink: &armdatafactory.BlobSink{
										Type: to.Ptr("BlobSink"),
									},
									Source: &armdatafactory.BlobSource{
										Type: to.Ptr("BlobSource"),
									},
								},
							}},
						IsSequential: to.Ptr(true),
						Items: &armdatafactory.Expression{
							Type:  to.Ptr(armdatafactory.ExpressionTypeExpression),
							Value: to.Ptr("@pipeline().parameters.OutputBlobNameList"),
						},
					},
				}},
			Parameters: map[string]*armdatafactory.ParameterSpecification{
				"JobId": {
					Type: to.Ptr(armdatafactory.ParameterTypeString),
				},
				"OutputBlobNameList": {
					Type: to.Ptr(armdatafactory.ParameterTypeArray),
				},
			},
			Policy: &armdatafactory.PipelinePolicy{
				ElapsedTimeMetric: &armdatafactory.PipelineElapsedTimeMetricPolicy{
					Duration: "0.00:10:00",
				},
			},
			RunDimensions: map[string]any{
				"JobId": map[string]any{
					"type":  "Expression",
					"value": "@pipeline().parameters.JobId",
				},
			},
			Variables: map[string]*armdatafactory.VariableSpecification{
				"TestVariableArray": {
					Type: to.Ptr(armdatafactory.VariableTypeArray),
				},
			},
		},
	}, &armdatafactory.PipelinesClientCreateOrUpdateOptions{IfMatch: nil})
	if err != nil {
		log.Fatalf("failed to finish the request: %v", err)
	}
	// You could use response here. We use blank identifier for just demo purposes.
	_ = res
	// If the HTTP response code is 200 as defined in example definition, your response structure would look as follows. Please pay attention that all the values in the output are fake values for just demo purposes.
	// res.PipelineResource = armdatafactory.PipelineResource{
	// 	Name: to.Ptr("examplePipeline"),
	// 	Type: to.Ptr("Microsoft.DataFactory/factories/pipelines"),
	// 	Etag: to.Ptr("0a0069d4-0000-0000-0000-5b245bd50000"),
	// 	ID: to.Ptr("/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline"),
	// 	Properties: &armdatafactory.Pipeline{
	// 		Activities: []armdatafactory.ActivityClassification{
	// 			&armdatafactory.ForEachActivity{
	// 				Name: to.Ptr("ExampleForeachActivity"),
	// 				Type: to.Ptr("ForEach"),
	// 				TypeProperties: &armdatafactory.ForEachActivityTypeProperties{
	// 					Activities: []armdatafactory.ActivityClassification{
	// 						&armdatafactory.CopyActivity{
	// 							Name: to.Ptr("ExampleCopyActivity"),
	// 							Type: to.Ptr("Copy"),
	// 							Inputs: []*armdatafactory.DatasetReference{
	// 								{
	// 									Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
	// 									Parameters: map[string]any{
	// 										"MyFileName": "examplecontainer.csv",
	// 										"MyFolderPath": "examplecontainer",
	// 									},
	// 									ReferenceName: to.Ptr("exampleDataset"),
	// 							}},
	// 							Outputs: []*armdatafactory.DatasetReference{
	// 								{
	// 									Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
	// 									Parameters: map[string]any{
	// 										"MyFileName": map[string]any{
	// 											"type": "Expression",
	// 											"value": "@item()",
	// 										},
	// 										"MyFolderPath": "examplecontainer",
	// 									},
	// 									ReferenceName: to.Ptr("exampleDataset"),
	// 							}},
	// 							TypeProperties: &armdatafactory.CopyActivityTypeProperties{
	// 								DataIntegrationUnits: float64(32),
	// 								Sink: &armdatafactory.BlobSink{
	// 									Type: to.Ptr("BlobSink"),
	// 								},
	// 								Source: &armdatafactory.BlobSource{
	// 									Type: to.Ptr("BlobSource"),
	// 								},
	// 							},
	// 					}},
	// 					IsSequential: to.Ptr(true),
	// 					Items: &armdatafactory.Expression{
	// 						Type: to.Ptr(armdatafactory.ExpressionTypeExpression),
	// 						Value: to.Ptr("@pipeline().parameters.OutputBlobNameList"),
	// 					},
	// 				},
	// 		}},
	// 		Parameters: map[string]*armdatafactory.ParameterSpecification{
	// 			"JobId": &armdatafactory.ParameterSpecification{
	// 				Type: to.Ptr(armdatafactory.ParameterTypeString),
	// 			},
	// 			"OutputBlobNameList": &armdatafactory.ParameterSpecification{
	// 				Type: to.Ptr(armdatafactory.ParameterTypeArray),
	// 			},
	// 		},
	// 		RunDimensions: map[string]any{
	// 			"JobId": map[string]any{
	// 				"type": "Expression",
	// 				"value": "@pipeline().parameters.JobId",
	// 			},
	// 		},
	// 		Variables: map[string]*armdatafactory.VariableSpecification{
	// 			"TestVariableArray": &armdatafactory.VariableSpecification{
	// 				Type: to.Ptr(armdatafactory.VariableTypeArray),
	// 			},
	// 		},
	// 	},
	// }
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

JavaScript

const { DataFactoryManagementClient } = require("@azure/arm-datafactory");
const { DefaultAzureCredential } = require("@azure/identity");

/**
 * This sample demonstrates how to Creates or updates a pipeline.
 *
 * @summary Creates or updates a pipeline.
 * x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Create.json
 */
async function pipelinesCreate() {
  const subscriptionId =
    process.env["DATAFACTORY_SUBSCRIPTION_ID"] || "12345678-1234-1234-1234-12345678abc";
  const resourceGroupName = process.env["DATAFACTORY_RESOURCE_GROUP"] || "exampleResourceGroup";
  const factoryName = "exampleFactoryName";
  const pipelineName = "examplePipeline";
  const pipeline = {
    activities: [
      {
        name: "ExampleForeachActivity",
        type: "ForEach",
        activities: [
          {
            name: "ExampleCopyActivity",
            type: "Copy",
            dataIntegrationUnits: 32,
            inputs: [
              {
                type: "DatasetReference",
                parameters: {
                  myFileName: "examplecontainer.csv",
                  myFolderPath: "examplecontainer",
                },
                referenceName: "exampleDataset",
              },
            ],
            outputs: [
              {
                type: "DatasetReference",
                parameters: {
                  myFileName: { type: "Expression", value: "@item()" },
                  myFolderPath: "examplecontainer",
                },
                referenceName: "exampleDataset",
              },
            ],
            sink: { type: "BlobSink" },
            source: { type: "BlobSource" },
          },
        ],
        isSequential: true,
        items: {
          type: "Expression",
          value: "@pipeline().parameters.OutputBlobNameList",
        },
      },
    ],
    parameters: {
      jobId: { type: "String" },
      outputBlobNameList: { type: "Array" },
    },
    policy: { elapsedTimeMetric: { duration: "0.00:10:00" } },
    runDimensions: {
      jobId: { type: "Expression", value: "@pipeline().parameters.JobId" },
    },
    variables: { testVariableArray: { type: "Array" } },
  };
  const credential = new DefaultAzureCredential();
  const client = new DataFactoryManagementClient(credential, subscriptionId);
  const result = await client.pipelines.createOrUpdate(
    resourceGroupName,
    factoryName,
    pipelineName,
    pipeline,
  );
  console.log(result);
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

dotnet

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Azure;
using Azure.Core;
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.DataFactory;
using Azure.ResourceManager.DataFactory.Models;

// Generated from example definition: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Create.json
// this example is just showing the usage of "Pipelines_CreateOrUpdate" operation, for the dependent resources, they will have to be created separately.

// get your azure access token, for more details of how Azure SDK get your access token, please refer to https://learn.microsoft.com/en-us/dotnet/azure/sdk/authentication?tabs=command-line
TokenCredential cred = new DefaultAzureCredential();
// authenticate your client
ArmClient client = new ArmClient(cred);

// this example assumes you already have this DataFactoryPipelineResource created on azure
// for more information of creating DataFactoryPipelineResource, please refer to the document of DataFactoryPipelineResource
string subscriptionId = "12345678-1234-1234-1234-12345678abc";
string resourceGroupName = "exampleResourceGroup";
string factoryName = "exampleFactoryName";
string pipelineName = "examplePipeline";
ResourceIdentifier dataFactoryPipelineResourceId = DataFactoryPipelineResource.CreateResourceIdentifier(subscriptionId, resourceGroupName, factoryName, pipelineName);
DataFactoryPipelineResource dataFactoryPipeline = client.GetDataFactoryPipelineResource(dataFactoryPipelineResourceId);

// invoke the operation
DataFactoryPipelineData data = new DataFactoryPipelineData()
{
    Activities =
    {
    new ForEachActivity("ExampleForeachActivity",new DataFactoryExpression(DataFactoryExpressionType.Expression,"@pipeline().parameters.OutputBlobNameList"),new PipelineActivity[]
    {
    new CopyActivity("ExampleCopyActivity",new DataFactoryBlobSource(),new DataFactoryBlobSink())
    {
    Inputs =
    {
    new DatasetReference(DatasetReferenceType.DatasetReference,"exampleDataset")
    {
    Parameters =
    {
    ["MyFileName"] = BinaryData.FromString("\"examplecontainer.csv\""),
    ["MyFolderPath"] = BinaryData.FromString("\"examplecontainer\""),
    },
    }
    },
    Outputs =
    {
    new DatasetReference(DatasetReferenceType.DatasetReference,"exampleDataset")
    {
    Parameters =
    {
    ["MyFileName"] = BinaryData.FromObjectAsJson(new Dictionary<string, object>()
    {
    ["type"] = "Expression",
    ["value"] = "@item()"}),
    ["MyFolderPath"] = BinaryData.FromString("\"examplecontainer\""),
    },
    }
    },
    DataIntegrationUnits = 32,
    }
    })
    {
    IsSequential = true,
    }
    },
    Parameters =
    {
    ["JobId"] = new EntityParameterSpecification(EntityParameterType.String),
    ["OutputBlobNameList"] = new EntityParameterSpecification(EntityParameterType.Array),
    },
    Variables =
    {
    ["TestVariableArray"] = new PipelineVariableSpecification(PipelineVariableType.Array),
    },
    RunDimensions =
    {
    ["JobId"] = BinaryData.FromObjectAsJson(new Dictionary<string, object>()
    {
    ["type"] = "Expression",
    ["value"] = "@pipeline().parameters.JobId"}),
    },
    ElapsedTimeMetricDuration = BinaryData.FromString("\"0.00:10:00\""),
};
ArmOperation<DataFactoryPipelineResource> lro = await dataFactoryPipeline.UpdateAsync(WaitUntil.Completed, data);
DataFactoryPipelineResource result = lro.Value;

// the variable result is a resource, you could call other operations on this instance as well
// but just for demo, we get its data from this resource instance
DataFactoryPipelineData resourceData = result.Data;
// for demo we just print out the id
Console.WriteLine($"Succeeded on id: {resourceData.Id}");

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Sample Response

Status code:

200

Date: Sat, 16 Jun 2018 00:37:41 GMT
X-Content-Type-Options: nosniff
x-ms-ratelimit-remaining-subscription-writes: 1192
x-ms-request-id: e4c589b7-a9fe-4c28-981c-3855ec27d264
x-ms-correlation-request-id: e4c589b7-a9fe-4c28-981c-3855ec27d264

Response Body

{
  "id": "/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline",
  "name": "examplePipeline",
  "type": "Microsoft.DataFactory/factories/pipelines",
  "properties": {
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      },
      "JobId": {
        "type": "String"
      }
    },
    "variables": {
      "TestVariableArray": {
        "type": "Array"
      }
    },
    "runDimensions": {
      "JobId": {
        "value": "@pipeline().parameters.JobId",
        "type": "Expression"
      }
    }
  },
  "etag": "0a0069d4-0000-0000-0000-5b245bd50000"
}

Pipelines_Update

Sample Request

HTTP

PUT https://management.azure.com/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline?api-version=2018-06-01

{
  "properties": {
    "description": "Example description",
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      }
    },
    "policy": {
      "elapsedTimeMetric": {
        "duration": "0.00:10:00"
      }
    }
  }
}

Java

import com.azure.core.management.serializer.SerializerFactory;
import com.azure.core.util.serializer.SerializerEncoding;
import com.azure.resourcemanager.datafactory.models.BlobSink;
import com.azure.resourcemanager.datafactory.models.BlobSource;
import com.azure.resourcemanager.datafactory.models.CopyActivity;
import com.azure.resourcemanager.datafactory.models.DatasetReference;
import com.azure.resourcemanager.datafactory.models.Expression;
import com.azure.resourcemanager.datafactory.models.ForEachActivity;
import com.azure.resourcemanager.datafactory.models.ParameterSpecification;
import com.azure.resourcemanager.datafactory.models.ParameterType;
import com.azure.resourcemanager.datafactory.models.PipelineElapsedTimeMetricPolicy;
import com.azure.resourcemanager.datafactory.models.PipelinePolicy;
import com.azure.resourcemanager.datafactory.models.PipelineResource;
import java.io.IOException;
import java.util.Arrays;
import java.util.HashMap;
import java.util.Map;

/**
 * Samples for Pipelines CreateOrUpdate.
 */
public final class Main {
    /*
     * x-ms-original-file:
     * specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Update.json
     */
    /**
     * Sample code: Pipelines_Update.
     * 
     * @param manager Entry point to DataFactoryManager.
     */
    public static void pipelinesUpdate(com.azure.resourcemanager.datafactory.DataFactoryManager manager)
        throws IOException {
        PipelineResource resource = manager.pipelines().getWithResponse("exampleResourceGroup", "exampleFactoryName",
            "examplePipeline", null, com.azure.core.util.Context.NONE).getValue();
        resource.update().withDescription("Example description").withActivities(Arrays.asList(new ForEachActivity()
            .withName("ExampleForeachActivity").withIsSequential(true)
            .withItems(new Expression().withValue("@pipeline().parameters.OutputBlobNameList"))
            .withActivities(Arrays.asList(new CopyActivity().withName("ExampleCopyActivity")
                .withInputs(Arrays.asList(new DatasetReference().withReferenceName("exampleDataset")
                    .withParameters(mapOf("MyFileName", "examplecontainer.csv", "MyFolderPath", "examplecontainer"))))
                .withOutputs(Arrays.asList(new DatasetReference().withReferenceName("exampleDataset")
                    .withParameters(mapOf("MyFileName",
                        SerializerFactory.createDefaultManagementSerializerAdapter().deserialize(
                            "{\"type\":\"Expression\",\"value\":\"@item()\"}", Object.class, SerializerEncoding.JSON),
                        "MyFolderPath", "examplecontainer"))))
                .withSource(new BlobSource()).withSink(new BlobSink()).withDataIntegrationUnits(32)))))
            .withParameters(mapOf("OutputBlobNameList", new ParameterSpecification().withType(ParameterType.ARRAY)))
            .withPolicy(new PipelinePolicy()
                .withElapsedTimeMetric(new PipelineElapsedTimeMetricPolicy().withDuration("0.00:10:00")))
            .apply();
    }

    // Use "Map.of" if available
    @SuppressWarnings("unchecked")
    private static <T> Map<String, T> mapOf(Object... inputs) {
        Map<String, T> map = new HashMap<>();
        for (int i = 0; i < inputs.length; i += 2) {
            String key = (String) inputs[i];
            T value = (T) inputs[i + 1];
            map.put(key, value);
        }
        return map;
    }
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Python

from azure.identity import DefaultAzureCredential
from azure.mgmt.datafactory import DataFactoryManagementClient

"""
# PREREQUISITES
    pip install azure-identity
    pip install azure-mgmt-datafactory
# USAGE
    python pipelines_update.py

    Before run the sample, please set the values of the client ID, tenant ID and client secret
    of the AAD application as environment variables: AZURE_CLIENT_ID, AZURE_TENANT_ID,
    AZURE_CLIENT_SECRET. For more info about how to get the value, please see:
    https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal
"""


def main():
    client = DataFactoryManagementClient(
        credential=DefaultAzureCredential(),
        subscription_id="12345678-1234-1234-1234-12345678abc",
    )

    response = client.pipelines.create_or_update(
        resource_group_name="exampleResourceGroup",
        factory_name="exampleFactoryName",
        pipeline_name="examplePipeline",
        pipeline={
            "properties": {
                "activities": [
                    {
                        "name": "ExampleForeachActivity",
                        "type": "ForEach",
                        "typeProperties": {
                            "activities": [
                                {
                                    "inputs": [
                                        {
                                            "parameters": {
                                                "MyFileName": "examplecontainer.csv",
                                                "MyFolderPath": "examplecontainer",
                                            },
                                            "referenceName": "exampleDataset",
                                            "type": "DatasetReference",
                                        }
                                    ],
                                    "name": "ExampleCopyActivity",
                                    "outputs": [
                                        {
                                            "parameters": {
                                                "MyFileName": {"type": "Expression", "value": "@item()"},
                                                "MyFolderPath": "examplecontainer",
                                            },
                                            "referenceName": "exampleDataset",
                                            "type": "DatasetReference",
                                        }
                                    ],
                                    "type": "Copy",
                                    "typeProperties": {
                                        "dataIntegrationUnits": 32,
                                        "sink": {"type": "BlobSink"},
                                        "source": {"type": "BlobSource"},
                                    },
                                }
                            ],
                            "isSequential": True,
                            "items": {"type": "Expression", "value": "@pipeline().parameters.OutputBlobNameList"},
                        },
                    }
                ],
                "description": "Example description",
                "parameters": {"OutputBlobNameList": {"type": "Array"}},
                "policy": {"elapsedTimeMetric": {"duration": "0.00:10:00"}},
            }
        },
    )
    print(response)


# x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Update.json
if __name__ == "__main__":
    main()

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Go

package armdatafactory_test

import (
	"context"
	"log"

	"github.com/Azure/azure-sdk-for-go/sdk/azcore/to"
	"github.com/Azure/azure-sdk-for-go/sdk/azidentity"
	"github.com/Azure/azure-sdk-for-go/sdk/resourcemanager/datafactory/armdatafactory/v6"
)

// Generated from example definition: https://github.com/Azure/azure-rest-api-specs/blob/01a71545e82bb98b8137d3038150c436d46a59ed/specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Update.json
func ExamplePipelinesClient_CreateOrUpdate_pipelinesUpdate() {
	cred, err := azidentity.NewDefaultAzureCredential(nil)
	if err != nil {
		log.Fatalf("failed to obtain a credential: %v", err)
	}
	ctx := context.Background()
	clientFactory, err := armdatafactory.NewClientFactory("<subscription-id>", cred, nil)
	if err != nil {
		log.Fatalf("failed to create client: %v", err)
	}
	res, err := clientFactory.NewPipelinesClient().CreateOrUpdate(ctx, "exampleResourceGroup", "exampleFactoryName", "examplePipeline", armdatafactory.PipelineResource{
		Properties: &armdatafactory.Pipeline{
			Description: to.Ptr("Example description"),
			Activities: []armdatafactory.ActivityClassification{
				&armdatafactory.ForEachActivity{
					Name: to.Ptr("ExampleForeachActivity"),
					Type: to.Ptr("ForEach"),
					TypeProperties: &armdatafactory.ForEachActivityTypeProperties{
						Activities: []armdatafactory.ActivityClassification{
							&armdatafactory.CopyActivity{
								Name: to.Ptr("ExampleCopyActivity"),
								Type: to.Ptr("Copy"),
								Inputs: []*armdatafactory.DatasetReference{
									{
										Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
										Parameters: map[string]any{
											"MyFileName":   "examplecontainer.csv",
											"MyFolderPath": "examplecontainer",
										},
										ReferenceName: to.Ptr("exampleDataset"),
									}},
								Outputs: []*armdatafactory.DatasetReference{
									{
										Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
										Parameters: map[string]any{
											"MyFileName": map[string]any{
												"type":  "Expression",
												"value": "@item()",
											},
											"MyFolderPath": "examplecontainer",
										},
										ReferenceName: to.Ptr("exampleDataset"),
									}},
								TypeProperties: &armdatafactory.CopyActivityTypeProperties{
									DataIntegrationUnits: float64(32),
									Sink: &armdatafactory.BlobSink{
										Type: to.Ptr("BlobSink"),
									},
									Source: &armdatafactory.BlobSource{
										Type: to.Ptr("BlobSource"),
									},
								},
							}},
						IsSequential: to.Ptr(true),
						Items: &armdatafactory.Expression{
							Type:  to.Ptr(armdatafactory.ExpressionTypeExpression),
							Value: to.Ptr("@pipeline().parameters.OutputBlobNameList"),
						},
					},
				}},
			Parameters: map[string]*armdatafactory.ParameterSpecification{
				"OutputBlobNameList": {
					Type: to.Ptr(armdatafactory.ParameterTypeArray),
				},
			},
			Policy: &armdatafactory.PipelinePolicy{
				ElapsedTimeMetric: &armdatafactory.PipelineElapsedTimeMetricPolicy{
					Duration: "0.00:10:00",
				},
			},
		},
	}, &armdatafactory.PipelinesClientCreateOrUpdateOptions{IfMatch: nil})
	if err != nil {
		log.Fatalf("failed to finish the request: %v", err)
	}
	// You could use response here. We use blank identifier for just demo purposes.
	_ = res
	// If the HTTP response code is 200 as defined in example definition, your response structure would look as follows. Please pay attention that all the values in the output are fake values for just demo purposes.
	// res.PipelineResource = armdatafactory.PipelineResource{
	// 	Name: to.Ptr("examplePipeline"),
	// 	Type: to.Ptr("Microsoft.DataFactory/factories/pipelines"),
	// 	Etag: to.Ptr("0a006cd4-0000-0000-0000-5b245bd60000"),
	// 	ID: to.Ptr("/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline"),
	// 	Properties: &armdatafactory.Pipeline{
	// 		Description: to.Ptr("Example description"),
	// 		Activities: []armdatafactory.ActivityClassification{
	// 			&armdatafactory.ForEachActivity{
	// 				Name: to.Ptr("ExampleForeachActivity"),
	// 				Type: to.Ptr("ForEach"),
	// 				TypeProperties: &armdatafactory.ForEachActivityTypeProperties{
	// 					Activities: []armdatafactory.ActivityClassification{
	// 						&armdatafactory.CopyActivity{
	// 							Name: to.Ptr("ExampleCopyActivity"),
	// 							Type: to.Ptr("Copy"),
	// 							Inputs: []*armdatafactory.DatasetReference{
	// 								{
	// 									Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
	// 									Parameters: map[string]any{
	// 										"MyFileName": "examplecontainer.csv",
	// 										"MyFolderPath": "examplecontainer",
	// 									},
	// 									ReferenceName: to.Ptr("exampleDataset"),
	// 							}},
	// 							Outputs: []*armdatafactory.DatasetReference{
	// 								{
	// 									Type: to.Ptr(armdatafactory.DatasetReferenceTypeDatasetReference),
	// 									Parameters: map[string]any{
	// 										"MyFileName": map[string]any{
	// 											"type": "Expression",
	// 											"value": "@item()",
	// 										},
	// 										"MyFolderPath": "examplecontainer",
	// 									},
	// 									ReferenceName: to.Ptr("exampleDataset"),
	// 							}},
	// 							TypeProperties: &armdatafactory.CopyActivityTypeProperties{
	// 								DataIntegrationUnits: float64(32),
	// 								Sink: &armdatafactory.BlobSink{
	// 									Type: to.Ptr("BlobSink"),
	// 								},
	// 								Source: &armdatafactory.BlobSource{
	// 									Type: to.Ptr("BlobSource"),
	// 								},
	// 							},
	// 					}},
	// 					IsSequential: to.Ptr(true),
	// 					Items: &armdatafactory.Expression{
	// 						Type: to.Ptr(armdatafactory.ExpressionTypeExpression),
	// 						Value: to.Ptr("@pipeline().parameters.OutputBlobNameList"),
	// 					},
	// 				},
	// 		}},
	// 		Parameters: map[string]*armdatafactory.ParameterSpecification{
	// 			"OutputBlobNameList": &armdatafactory.ParameterSpecification{
	// 				Type: to.Ptr(armdatafactory.ParameterTypeArray),
	// 			},
	// 		},
	// 	},
	// }
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

JavaScript

const { DataFactoryManagementClient } = require("@azure/arm-datafactory");
const { DefaultAzureCredential } = require("@azure/identity");

/**
 * This sample demonstrates how to Creates or updates a pipeline.
 *
 * @summary Creates or updates a pipeline.
 * x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Update.json
 */
async function pipelinesUpdate() {
  const subscriptionId =
    process.env["DATAFACTORY_SUBSCRIPTION_ID"] || "12345678-1234-1234-1234-12345678abc";
  const resourceGroupName = process.env["DATAFACTORY_RESOURCE_GROUP"] || "exampleResourceGroup";
  const factoryName = "exampleFactoryName";
  const pipelineName = "examplePipeline";
  const pipeline = {
    description: "Example description",
    activities: [
      {
        name: "ExampleForeachActivity",
        type: "ForEach",
        activities: [
          {
            name: "ExampleCopyActivity",
            type: "Copy",
            dataIntegrationUnits: 32,
            inputs: [
              {
                type: "DatasetReference",
                parameters: {
                  myFileName: "examplecontainer.csv",
                  myFolderPath: "examplecontainer",
                },
                referenceName: "exampleDataset",
              },
            ],
            outputs: [
              {
                type: "DatasetReference",
                parameters: {
                  myFileName: { type: "Expression", value: "@item()" },
                  myFolderPath: "examplecontainer",
                },
                referenceName: "exampleDataset",
              },
            ],
            sink: { type: "BlobSink" },
            source: { type: "BlobSource" },
          },
        ],
        isSequential: true,
        items: {
          type: "Expression",
          value: "@pipeline().parameters.OutputBlobNameList",
        },
      },
    ],
    parameters: { outputBlobNameList: { type: "Array" } },
    policy: { elapsedTimeMetric: { duration: "0.00:10:00" } },
  };
  const credential = new DefaultAzureCredential();
  const client = new DataFactoryManagementClient(credential, subscriptionId);
  const result = await client.pipelines.createOrUpdate(
    resourceGroupName,
    factoryName,
    pipelineName,
    pipeline,
  );
  console.log(result);
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

dotnet

using System;
using System.Collections.Generic;
using System.Threading.Tasks;
using Azure;
using Azure.Core;
using Azure.Identity;
using Azure.ResourceManager;
using Azure.ResourceManager.DataFactory;
using Azure.ResourceManager.DataFactory.Models;

// Generated from example definition: specification/datafactory/resource-manager/Microsoft.DataFactory/stable/2018-06-01/examples/Pipelines_Update.json
// this example is just showing the usage of "Pipelines_CreateOrUpdate" operation, for the dependent resources, they will have to be created separately.

// get your azure access token, for more details of how Azure SDK get your access token, please refer to https://learn.microsoft.com/en-us/dotnet/azure/sdk/authentication?tabs=command-line
TokenCredential cred = new DefaultAzureCredential();
// authenticate your client
ArmClient client = new ArmClient(cred);

// this example assumes you already have this DataFactoryPipelineResource created on azure
// for more information of creating DataFactoryPipelineResource, please refer to the document of DataFactoryPipelineResource
string subscriptionId = "12345678-1234-1234-1234-12345678abc";
string resourceGroupName = "exampleResourceGroup";
string factoryName = "exampleFactoryName";
string pipelineName = "examplePipeline";
ResourceIdentifier dataFactoryPipelineResourceId = DataFactoryPipelineResource.CreateResourceIdentifier(subscriptionId, resourceGroupName, factoryName, pipelineName);
DataFactoryPipelineResource dataFactoryPipeline = client.GetDataFactoryPipelineResource(dataFactoryPipelineResourceId);

// invoke the operation
DataFactoryPipelineData data = new DataFactoryPipelineData()
{
    Description = "Example description",
    Activities =
    {
    new ForEachActivity("ExampleForeachActivity",new DataFactoryExpression(DataFactoryExpressionType.Expression,"@pipeline().parameters.OutputBlobNameList"),new PipelineActivity[]
    {
    new CopyActivity("ExampleCopyActivity",new DataFactoryBlobSource(),new DataFactoryBlobSink())
    {
    Inputs =
    {
    new DatasetReference(DatasetReferenceType.DatasetReference,"exampleDataset")
    {
    Parameters =
    {
    ["MyFileName"] = BinaryData.FromString("\"examplecontainer.csv\""),
    ["MyFolderPath"] = BinaryData.FromString("\"examplecontainer\""),
    },
    }
    },
    Outputs =
    {
    new DatasetReference(DatasetReferenceType.DatasetReference,"exampleDataset")
    {
    Parameters =
    {
    ["MyFileName"] = BinaryData.FromObjectAsJson(new Dictionary<string, object>()
    {
    ["type"] = "Expression",
    ["value"] = "@item()"}),
    ["MyFolderPath"] = BinaryData.FromString("\"examplecontainer\""),
    },
    }
    },
    DataIntegrationUnits = 32,
    }
    })
    {
    IsSequential = true,
    }
    },
    Parameters =
    {
    ["OutputBlobNameList"] = new EntityParameterSpecification(EntityParameterType.Array),
    },
    ElapsedTimeMetricDuration = BinaryData.FromString("\"0.00:10:00\""),
};
ArmOperation<DataFactoryPipelineResource> lro = await dataFactoryPipeline.UpdateAsync(WaitUntil.Completed, data);
DataFactoryPipelineResource result = lro.Value;

// the variable result is a resource, you could call other operations on this instance as well
// but just for demo, we get its data from this resource instance
DataFactoryPipelineData resourceData = result.Data;
// for demo we just print out the id
Console.WriteLine($"Succeeded on id: {resourceData.Id}");

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Sample Response

Status code:

200

Date: Sat, 16 Jun 2018 00:37:43 GMT
X-Content-Type-Options: nosniff
x-ms-ratelimit-remaining-subscription-writes: 1191
x-ms-request-id: 91a763b0-d944-493f-83fc-e989c8b6e122
x-ms-correlation-request-id: 91a763b0-d944-493f-83fc-e989c8b6e122

Response Body

{
  "id": "/subscriptions/12345678-1234-1234-1234-12345678abc/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/pipelines/examplePipeline",
  "name": "examplePipeline",
  "type": "Microsoft.DataFactory/factories/pipelines",
  "properties": {
    "description": "Example description",
    "activities": [
      {
        "type": "ForEach",
        "typeProperties": {
          "isSequential": true,
          "items": {
            "value": "@pipeline().parameters.OutputBlobNameList",
            "type": "Expression"
          },
          "activities": [
            {
              "type": "Copy",
              "typeProperties": {
                "source": {
                  "type": "BlobSource"
                },
                "sink": {
                  "type": "BlobSink"
                },
                "dataIntegrationUnits": 32
              },
              "inputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": "examplecontainer.csv"
                  },
                  "type": "DatasetReference"
                }
              ],
              "outputs": [
                {
                  "referenceName": "exampleDataset",
                  "parameters": {
                    "MyFolderPath": "examplecontainer",
                    "MyFileName": {
                      "value": "@item()",
                      "type": "Expression"
                    }
                  },
                  "type": "DatasetReference"
                }
              ],
              "name": "ExampleCopyActivity"
            }
          ]
        },
        "name": "ExampleForeachActivity"
      }
    ],
    "parameters": {
      "OutputBlobNameList": {
        "type": "Array"
      }
    }
  },
  "etag": "0a006cd4-0000-0000-0000-5b245bd60000"
}

Definitions

Name	Description
ActivityDependency	Activity dependency information.
ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
ActivityPolicy	Execution policy for an activity.
ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
AppendVariableActivity	Append value for a Variable of type Array.
AzureDataExplorerCommandActivity	Azure Data Explorer command activity.
AzureFunctionActivity	Azure Function activity.
AzureFunctionActivityMethod	The list of HTTP methods supported by a AzureFunctionActivity.
AzureKeyVaultSecretReference	Azure Key Vault secret reference.
AzureMLBatchExecutionActivity	Azure ML Batch Execution activity.
AzureMLExecutePipelineActivity	Azure ML Execute Pipeline activity.
AzureMLUpdateResourceActivity	Azure ML Update Resource management activity.
AzureMLWebServiceFile	Azure ML WebService Input/Output file
BigDataPoolParametrizationReference	Big data pool reference type.
BigDataPoolReferenceType	Big data pool reference type.
CloudError	The object that defines the structure of an Azure Data Factory error response.
Compute	Compute properties for data flow activity.
ConfigurationType	The type of the spark config.
ControlActivity	Base class for all control activities like IfCondition, ForEach , Until.
CopyActivity	Copy activity.
CopyActivityLogSettings	Settings for copy activity log.
CredentialReference	Credential reference type.
CredentialReferenceType	Credential reference type.
CustomActivity	Custom activity type.
CustomActivityReferenceObject	Reference objects for custom activity
DatabricksNotebookActivity	DatabricksNotebook activity.
DatabricksSparkJarActivity	DatabricksSparkJar activity.
DatabricksSparkPythonActivity	DatabricksSparkPython activity.
DataFlowReference	Data flow reference type.
DataFlowReferenceType	Data flow reference type.
DataFlowStagingInfo	Staging info for execute data flow activity.
DataLakeAnalyticsUSQLActivity	Data Lake Analytics U-SQL activity.
DatasetReference	Dataset reference type.
DeleteActivity	Delete activity.
DependencyCondition	Match-Condition for the dependency.
ExecuteDataFlowActivity	Execute data flow activity.
ExecutePipelineActivity	Execute pipeline activity.
ExecutePipelineActivityPolicy	Execution policy for an execute pipeline activity.
ExecuteSSISPackageActivity	Execute SSIS package activity.
ExecuteWranglingDataflowActivity	Execute power query activity.
ExecutionActivity	Base class for all execution activities.
Expression	Azure Data Factory expression definition.
FailActivity	This activity will fail within its own scope and output a custom error message and error code. The error message and code can provided either as a string literal or as an expression that can be evaluated to a string at runtime. The activity scope can be the whole pipeline or a control activity (e.g. foreach, switch, until), if the fail activity is contained in it.
FilterActivity	Filter and return results from input array based on the conditions.
Folder	The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.
ForEachActivity	This activity is used for iterating over a collection and execute given activities.
GetMetadataActivity	Activity to get metadata of dataset
HDInsightActivityDebugInfoOption	The HDInsightActivityDebugInfoOption settings to use.
HDInsightHiveActivity	HDInsight Hive activity type.
HDInsightMapReduceActivity	HDInsight MapReduce activity type.
HDInsightPigActivity	HDInsight Pig activity type.
HDInsightSparkActivity	HDInsight Spark activity.
HDInsightStreamingActivity	HDInsight streaming activity type.
IfConditionActivity	This activity evaluates a boolean expression and executes either the activities under the ifTrueActivities property or the ifFalseActivities property depending on the result of the expression.
IntegrationRuntimeReference	Integration runtime reference type.
LinkedServiceReference	Linked service reference type.
LogLocationSettings	Log location settings.
LogSettings	Log settings.
LogStorageSettings	(Deprecated. Please use LogSettings) Log storage settings.
LookupActivity	Lookup activity.
NotebookParameter	Notebook parameter.
NotebookParameterType	Notebook parameter type.
NotebookReferenceType	Synapse notebook reference type.
ParameterSpecification	Definition of a single parameter for an entity.
ParameterType	Parameter type.
PipelineElapsedTimeMetricPolicy	Pipeline ElapsedTime Metric Policy.
PipelinePolicy	Pipeline Policy.
PipelineReference	Pipeline reference type.
PipelineResource	Pipeline resource type.
PowerQuerySink	Power query sink.
PowerQuerySinkMapping	Map Power Query mashup query to sink dataset(s).
RedirectIncompatibleRowSettings	Redirect incompatible row settings
ScriptActivity	Script activity type.
ScriptActivityLogDestination	The destination of logs. Type: string.
ScriptActivityParameter	Parameters of a script block.
ScriptActivityParameterDirection	The direction of the parameter.
ScriptActivityParameterType	The type of the parameter.
ScriptActivityScriptBlock	Script block of scripts.
ScriptActivityTypeProperties.LogSettings	Log settings of script activity.
SecureInputOutputPolicy	Execution policy for an activity that supports secure input and output.
SecureString	Azure Data Factory secure string definition. The string value will be masked with asterisks '*' during Get or List API calls.
SetVariableActivity	Set value for a Variable.
SkipErrorFile	Skip error file.
SparkConfigurationParametrizationReference	Spark configuration reference.
SparkConfigurationReferenceType	Spark configuration reference type.
SparkJobReferenceType	Synapse spark job reference type.
SqlServerStoredProcedureActivity	SQL stored procedure activity type.
SSISAccessCredential	SSIS access credential.
SSISChildPackage	SSIS embedded child package.
SSISExecutionCredential	SSIS package execution credential.
SSISExecutionParameter	SSIS execution parameter.
SSISLogLocation	SSIS package execution log location
SsisLogLocationType	The type of SSIS log location.
SSISPackageLocation	SSIS package location.
SsisPackageLocationType	The type of SSIS package location.
SSISPropertyOverride	SSIS property override.
StagingSettings	Staging settings.
SwitchActivity	This activity evaluates an expression and executes activities under the cases property that correspond to the expression evaluation expected in the equals property.
SwitchCase	Switch cases with have a value and corresponding activities.
SynapseNotebookActivity	Execute Synapse notebook activity.
SynapseNotebookReference	Synapse notebook reference type.
SynapseSparkJobDefinitionActivity	Execute spark job activity.
SynapseSparkJobReference	Synapse spark job reference type.
Type	Linked service reference type.
UntilActivity	This activity executes inner activities until the specified boolean expression results to true or timeout is reached, whichever is earlier.
UserProperty	User property.
ValidationActivity	This activity verifies that an external resource exists.
VariableSpecification	Definition of a single variable for a Pipeline.
VariableType	Variable type.
WaitActivity	This activity suspends pipeline execution for the specified interval.
WebActivity	Web activity.
WebActivityAuthentication	Web activity authentication properties.
WebActivityMethod	The list of HTTP methods supported by a WebActivity.
WebHookActivity	WebHook activity.
WebHookActivityMethod	The list of HTTP methods supported by a WebHook activity.

ActivityDependency

Activity dependency information.

Name	Type	Description
activity	string	Activity name.
dependencyConditions	DependencyCondition[]	Match-Condition for the dependency.

ActivityOnInactiveMarkAs

Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.

Name	Type	Description
Failed	string
Skipped	string
Succeeded	string

ActivityPolicy

Execution policy for an activity.

Name	Type	Description
retry	object	Maximum ordinary retry attempts. Default is 0. Type: integer (or Expression with resultType integer), minimum: 0.
retryIntervalInSeconds	integer	Interval between each retry attempt (in seconds). The default is 30 sec.
secureInput	boolean	When set to true, Input from activity is considered as secure and will not be logged to monitoring.
secureOutput	boolean	When set to true, Output from activity is considered as secure and will not be logged to monitoring.
timeout	object	Specifies the timeout for the activity to run. The default timeout is 7 days. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

ActivityState

Activity state. This is an optional property and if not provided, the state will be Active by default.

Name	Type	Description
Active	string
Inactive	string

AppendVariableActivity

Append value for a Variable of type Array.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: AppendVariable	Type of activity.
typeProperties.value	object	Value to be appended. Type: could be a static value matching type of the variable item or Expression with resultType matching type of the variable item
typeProperties.variableName	string	Name of the variable whose value needs to be appended to.
userProperties	UserProperty[]	Activity user properties.

AzureDataExplorerCommandActivity

Azure Data Explorer command activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: AzureDataExplorerCommand	Type of activity.
typeProperties.command	object	A control command, according to the Azure Data Explorer command syntax. Type: string (or Expression with resultType string).
typeProperties.commandTimeout	object	Control command timeout. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9]))..)
userProperties	UserProperty[]	Activity user properties.

AzureFunctionActivity

Azure Function activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: AzureFunctionActivity	Type of activity.
typeProperties.body	object	Represents the payload that will be sent to the endpoint. Required for POST/PUT method, not allowed for GET method Type: string (or Expression with resultType string).
typeProperties.functionName	object	Name of the Function that the Azure Function Activity will call. Type: string (or Expression with resultType string)
typeProperties.headers	object	Represents the headers that will be sent to the request. For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. Type: string (or Expression with resultType string).
typeProperties.method	AzureFunctionActivityMethod	Rest API method for target endpoint.
userProperties	UserProperty[]	Activity user properties.

AzureFunctionActivityMethod

The list of HTTP methods supported by a AzureFunctionActivity.

Name	Type	Description
DELETE	string
GET	string
HEAD	string
OPTIONS	string
POST	string
PUT	string
TRACE	string

AzureKeyVaultSecretReference

Azure Key Vault secret reference.

Name	Type	Description
secretName	object	The name of the secret in Azure Key Vault. Type: string (or Expression with resultType string).
secretVersion	object	The version of the secret in Azure Key Vault. The default value is the latest version of the secret. Type: string (or Expression with resultType string).
store	LinkedServiceReference	The Azure Key Vault linked service reference.
type	string: AzureKeyVaultSecret	Type of the secret.

AzureMLBatchExecutionActivity

Azure ML Batch Execution activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: AzureMLBatchExecution	Type of activity.
typeProperties.globalParameters	object	Key,Value pairs to be passed to the Azure ML Batch Execution Service endpoint. Keys must match the names of web service parameters defined in the published Azure ML web service. Values will be passed in the GlobalParameters property of the Azure ML batch execution request.
typeProperties.webServiceInputs	<string,  AzureMLWebServiceFile>	Key,Value pairs, mapping the names of Azure ML endpoint's Web Service Inputs to AzureMLWebServiceFile objects specifying the input Blob locations.. This information will be passed in the WebServiceInputs property of the Azure ML batch execution request.
typeProperties.webServiceOutputs	<string,  AzureMLWebServiceFile>	Key,Value pairs, mapping the names of Azure ML endpoint's Web Service Outputs to AzureMLWebServiceFile objects specifying the output Blob locations. This information will be passed in the WebServiceOutputs property of the Azure ML batch execution request.
userProperties	UserProperty[]	Activity user properties.

AzureMLExecutePipelineActivity

Azure ML Execute Pipeline activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: AzureMLExecutePipeline	Type of activity.
typeProperties.continueOnStepFailure	object	Whether to continue execution of other steps in the PipelineRun if a step fails. This information will be passed in the continueOnStepFailure property of the published pipeline execution request. Type: boolean (or Expression with resultType boolean).
typeProperties.dataPathAssignments	object	Dictionary used for changing data path assignments without retraining. Values will be passed in the dataPathAssignments property of the published pipeline execution request. Type: object (or Expression with resultType object).
typeProperties.experimentName	object	Run history experiment name of the pipeline run. This information will be passed in the ExperimentName property of the published pipeline execution request. Type: string (or Expression with resultType string).
typeProperties.mlParentRunId	object	The parent Azure ML Service pipeline run id. This information will be passed in the ParentRunId property of the published pipeline execution request. Type: string (or Expression with resultType string).
typeProperties.mlPipelineEndpointId	object	ID of the published Azure ML pipeline endpoint. Type: string (or Expression with resultType string).
typeProperties.mlPipelineId	object	ID of the published Azure ML pipeline. Type: string (or Expression with resultType string).
typeProperties.mlPipelineParameters	object	Key,Value pairs to be passed to the published Azure ML pipeline endpoint. Keys must match the names of pipeline parameters defined in the published pipeline. Values will be passed in the ParameterAssignments property of the published pipeline execution request. Type: object with key value pairs (or Expression with resultType object).
typeProperties.version	object	Version of the published Azure ML pipeline endpoint. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

AzureMLUpdateResourceActivity

Azure ML Update Resource management activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: AzureMLUpdateResource	Type of activity.
typeProperties.trainedModelFilePath	object	The relative file path in trainedModelLinkedService to represent the .ilearner file that will be uploaded by the update operation. Type: string (or Expression with resultType string).
typeProperties.trainedModelLinkedServiceName	LinkedServiceReference	Name of Azure Storage linked service holding the .ilearner file that will be uploaded by the update operation.
typeProperties.trainedModelName	object	Name of the Trained Model module in the Web Service experiment to be updated. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

AzureMLWebServiceFile

Azure ML WebService Input/Output file

Name	Type	Description
filePath	object	The relative file path, including container name, in the Azure Blob Storage specified by the LinkedService. Type: string (or Expression with resultType string).
linkedServiceName	LinkedServiceReference	Reference to an Azure Storage LinkedService, where Azure ML WebService Input/Output file located.

BigDataPoolParametrizationReference

Big data pool reference type.

Name	Type	Description
referenceName	object	Reference big data pool name. Type: string (or Expression with resultType string).
type	BigDataPoolReferenceType	Big data pool reference type.

BigDataPoolReferenceType

Big data pool reference type.

Name	Type	Description
BigDataPoolReference	string

CloudError

The object that defines the structure of an Azure Data Factory error response.

Name	Type	Description
error.code	string	Error code.
error.details	CloudError[]	Array with additional error details.
error.message	string	Error message.
error.target	string	Property name/path in request associated with error.

Compute

Compute properties for data flow activity.

Name	Type	Description
computeType	object	Compute type of the cluster which will execute data flow job. Possible values include: 'General', 'MemoryOptimized', 'ComputeOptimized'. Type: string (or Expression with resultType string)
coreCount	object	Core count of the cluster which will execute data flow job. Supported values are: 8, 16, 32, 48, 80, 144 and 272. Type: integer (or Expression with resultType integer)

ConfigurationType

The type of the spark config.

Name	Type	Description
Artifact	string
Customized	string
Default	string

ControlActivity

Base class for all control activities like IfCondition, ForEach , Until.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Container	Type of activity.
userProperties	UserProperty[]	Activity user properties.

CopyActivity

Copy activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
inputs	DatasetReference[]	List of inputs for the activity.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
outputs	DatasetReference[]	List of outputs for the activity.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Copy	Type of activity.
typeProperties.dataIntegrationUnits	object	Maximum number of data integration units that can be used to perform this data movement. Type: integer (or Expression with resultType integer), minimum: 0.
typeProperties.enableSkipIncompatibleRow	object	Whether to skip incompatible row. Default value is false. Type: boolean (or Expression with resultType boolean).
typeProperties.enableStaging	object	Specifies whether to copy data via an interim staging. Default value is false. Type: boolean (or Expression with resultType boolean).
typeProperties.logSettings	LogSettings	Log settings customer needs provide when enabling log.
typeProperties.logStorageSettings	LogStorageSettings	(Deprecated. Please use LogSettings) Log storage settings customer need to provide when enabling session log.
typeProperties.parallelCopies	object	Maximum number of concurrent sessions opened on the source or sink to avoid overloading the data store. Type: integer (or Expression with resultType integer), minimum: 0.
typeProperties.preserve	object[]	Preserve rules.
typeProperties.preserveRules	object[]	Preserve Rules.
typeProperties.redirectIncompatibleRowSettings	RedirectIncompatibleRowSettings	Redirect incompatible row settings when EnableSkipIncompatibleRow is true.
typeProperties.sink	CopySink	Copy activity sink.
typeProperties.skipErrorFile	SkipErrorFile	Specify the fault tolerance for data consistency.
typeProperties.source	CopySource	Copy activity source.
typeProperties.stagingSettings	StagingSettings	Specifies interim staging settings when EnableStaging is true.
typeProperties.translator	object	Copy activity translator. If not specified, tabular translator is used.
typeProperties.validateDataConsistency	object	Whether to enable Data Consistency validation. Type: boolean (or Expression with resultType boolean).
userProperties	UserProperty[]	Activity user properties.

CopyActivityLogSettings

Settings for copy activity log.

Name	Type	Description
enableReliableLogging	object	Specifies whether to enable reliable logging. Type: boolean (or Expression with resultType boolean).
logLevel	object	Gets or sets the log level, support: Info, Warning. Type: string (or Expression with resultType string).

CredentialReference

Credential reference type.

Name	Type	Description
referenceName	string	Reference credential name.
type	CredentialReferenceType	Credential reference type.

CredentialReferenceType

Credential reference type.

Name	Type	Description
CredentialReference	string

CustomActivity

Custom activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Custom	Type of activity.
typeProperties.autoUserSpecification	object	Elevation level and scope for the user, default is nonadmin task. Type: string (or Expression with resultType double).
typeProperties.command	object	Command for custom activity Type: string (or Expression with resultType string).
typeProperties.extendedProperties	object	User defined property bag. There is no restriction on the keys or values that can be used. The user specified custom activity has the full responsibility to consume and interpret the content defined.
typeProperties.folderPath	object	Folder path for resource files Type: string (or Expression with resultType string).
typeProperties.referenceObjects	CustomActivityReferenceObject	Reference objects
typeProperties.resourceLinkedService	LinkedServiceReference	Resource linked service reference.
typeProperties.retentionTimeInDays	object	The retention time for the files submitted for custom activity. Type: double (or Expression with resultType double).
userProperties	UserProperty[]	Activity user properties.

CustomActivityReferenceObject

Reference objects for custom activity

Name	Type	Description
datasets	DatasetReference[]	Dataset references.
linkedServices	LinkedServiceReference[]	Linked service references.

DatabricksNotebookActivity

DatabricksNotebook activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: DatabricksNotebook	Type of activity.
typeProperties.baseParameters	object	Base parameters to be used for each run of this job.If the notebook takes a parameter that is not specified, the default value from the notebook will be used.
typeProperties.libraries	object[]	A list of libraries to be installed on the cluster that will execute the job.
typeProperties.notebookPath	object	The absolute path of the notebook to be run in the Databricks Workspace. This path must begin with a slash. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

DatabricksSparkJarActivity

DatabricksSparkJar activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: DatabricksSparkJar	Type of activity.
typeProperties.libraries	object[]	A list of libraries to be installed on the cluster that will execute the job.
typeProperties.mainClassName	object	The full name of the class containing the main method to be executed. This class must be contained in a JAR provided as a library. Type: string (or Expression with resultType string).
typeProperties.parameters	object[]	Parameters that will be passed to the main method.
userProperties	UserProperty[]	Activity user properties.

DatabricksSparkPythonActivity

DatabricksSparkPython activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: DatabricksSparkPython	Type of activity.
typeProperties.libraries	object[]	A list of libraries to be installed on the cluster that will execute the job.
typeProperties.parameters	object[]	Command line parameters that will be passed to the Python file.
typeProperties.pythonFile	object	The URI of the Python file to be executed. DBFS paths are supported. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

DataFlowReference

Data flow reference type.

Name	Type	Description
datasetParameters	object	Reference data flow parameters from dataset.
parameters	object	Data flow parameters
referenceName	string	Reference data flow name.
type	DataFlowReferenceType	Data flow reference type.

DataFlowReferenceType

Data flow reference type.

Name	Type	Description
DataFlowReference	string

DataFlowStagingInfo

Staging info for execute data flow activity.

Name	Type	Description
folderPath	object	Folder path for staging blob. Type: string (or Expression with resultType string)
linkedService	LinkedServiceReference	Staging linked service reference.

DataLakeAnalyticsUSQLActivity

Data Lake Analytics U-SQL activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: DataLakeAnalyticsU-SQL	Type of activity.
typeProperties.compilationMode	object	Compilation mode of U-SQL. Must be one of these values : Semantic, Full and SingleBox. Type: string (or Expression with resultType string).
typeProperties.degreeOfParallelism	object	The maximum number of nodes simultaneously used to run the job. Default value is 1. Type: integer (or Expression with resultType integer), minimum: 1.
typeProperties.parameters	object	Parameters for U-SQL job request.
typeProperties.priority	object	Determines which jobs out of all that are queued should be selected to run first. The lower the number, the higher the priority. Default value is 1000. Type: integer (or Expression with resultType integer), minimum: 1.
typeProperties.runtimeVersion	object	Runtime version of the U-SQL engine to use. Type: string (or Expression with resultType string).
typeProperties.scriptLinkedService	LinkedServiceReference	Script linked service reference.
typeProperties.scriptPath	object	Case-sensitive path to folder that contains the U-SQL script. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

DatasetReference

Dataset reference type.

Name	Type	Description
parameters	object	Arguments for dataset.
referenceName	string	Reference dataset name.
type	enum: DatasetReference	Dataset reference type.

DeleteActivity

Delete activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Delete	Type of activity.
typeProperties.dataset	DatasetReference	Delete activity dataset reference.
typeProperties.enableLogging	object	Whether to record detailed logs of delete-activity execution. Default value is false. Type: boolean (or Expression with resultType boolean).
typeProperties.logStorageSettings	LogStorageSettings	Log storage settings customer need to provide when enableLogging is true.
typeProperties.maxConcurrentConnections	integer	The max concurrent connections to connect data source at the same time.
typeProperties.recursive	object	If true, files or sub-folders under current folder path will be deleted recursively. Default is false. Type: boolean (or Expression with resultType boolean).
typeProperties.storeSettings	StoreReadSettings	Delete activity store settings.
userProperties	UserProperty[]	Activity user properties.

DependencyCondition

Match-Condition for the dependency.

Name	Type	Description
Completed	string
Failed	string
Skipped	string
Succeeded	string

ExecuteDataFlowActivity

Execute data flow activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: ExecuteDataFlow	Type of activity.
typeProperties.compute	Compute	Compute properties for data flow activity.
typeProperties.continueOnError	object	Continue on error setting used for data flow execution. Enables processing to continue if a sink fails. Type: boolean (or Expression with resultType boolean)
typeProperties.dataFlow	DataFlowReference	Data flow reference.
typeProperties.integrationRuntime	IntegrationRuntimeReference	The integration runtime reference.
typeProperties.runConcurrently	object	Concurrent run setting used for data flow execution. Allows sinks with the same save order to be processed concurrently. Type: boolean (or Expression with resultType boolean)
typeProperties.sourceStagingConcurrency	object	Specify number of parallel staging for sources applicable to the sink. Type: integer (or Expression with resultType integer)
typeProperties.staging	DataFlowStagingInfo	Staging info for execute data flow activity.
typeProperties.traceLevel	object	Trace level setting used for data flow monitoring output. Supported values are: 'coarse', 'fine', and 'none'. Type: string (or Expression with resultType string)
userProperties	UserProperty[]	Activity user properties.

ExecutePipelineActivity

Execute pipeline activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ExecutePipelineActivityPolicy	Execute pipeline activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: ExecutePipeline	Type of activity.
typeProperties.parameters	object	Pipeline parameters.
typeProperties.pipeline	PipelineReference	Pipeline reference.
typeProperties.waitOnCompletion	boolean	Defines whether activity execution will wait for the dependent pipeline execution to finish. Default is false.
userProperties	UserProperty[]	Activity user properties.

ExecutePipelineActivityPolicy

Execution policy for an execute pipeline activity.

Name	Type	Description
secureInput	boolean	When set to true, Input from activity is considered as secure and will not be logged to monitoring.

ExecuteSSISPackageActivity

Execute SSIS package activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: ExecuteSSISPackage	Type of activity.
typeProperties.connectVia	IntegrationRuntimeReference	The integration runtime reference.
typeProperties.environmentPath	object	The environment path to execute the SSIS package. Type: string (or Expression with resultType string).
typeProperties.executionCredential	SSISExecutionCredential	The package execution credential.
typeProperties.logLocation	SSISLogLocation	SSIS package execution log location.
typeProperties.loggingLevel	object	The logging level of SSIS package execution. Type: string (or Expression with resultType string).
typeProperties.packageConnectionManagers	object	The package level connection managers to execute the SSIS package.
typeProperties.packageLocation	SSISPackageLocation	SSIS package location.
typeProperties.packageParameters	<string,  SSISExecutionParameter>	The package level parameters to execute the SSIS package.
typeProperties.projectConnectionManagers	object	The project level connection managers to execute the SSIS package.
typeProperties.projectParameters	<string,  SSISExecutionParameter>	The project level parameters to execute the SSIS package.
typeProperties.propertyOverrides	<string,  SSISPropertyOverride>	The property overrides to execute the SSIS package.
typeProperties.runtime	object	Specifies the runtime to execute SSIS package. The value should be "x86" or "x64". Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

ExecuteWranglingDataflowActivity

Execute power query activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: ExecuteWranglingDataflow	Type of activity.
typeProperties.compute	Compute	Compute properties for data flow activity.
typeProperties.continueOnError	object	Continue on error setting used for data flow execution. Enables processing to continue if a sink fails. Type: boolean (or Expression with resultType boolean)
typeProperties.dataFlow	DataFlowReference	Data flow reference.
typeProperties.integrationRuntime	IntegrationRuntimeReference	The integration runtime reference.
typeProperties.queries	PowerQuerySinkMapping[]	List of mapping for Power Query mashup query to sink dataset(s).
typeProperties.runConcurrently	object	Concurrent run setting used for data flow execution. Allows sinks with the same save order to be processed concurrently. Type: boolean (or Expression with resultType boolean)
typeProperties.sinks	<string,  PowerQuerySink>	(Deprecated. Please use Queries). List of Power Query activity sinks mapped to a queryName.
typeProperties.sourceStagingConcurrency	object	Specify number of parallel staging for sources applicable to the sink. Type: integer (or Expression with resultType integer)
typeProperties.staging	DataFlowStagingInfo	Staging info for execute data flow activity.
typeProperties.traceLevel	object	Trace level setting used for data flow monitoring output. Supported values are: 'coarse', 'fine', and 'none'. Type: string (or Expression with resultType string)
userProperties	UserProperty[]	Activity user properties.

ExecutionActivity

Base class for all execution activities.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Execution	Type of activity.
userProperties	UserProperty[]	Activity user properties.

Expression

Azure Data Factory expression definition.

Name	Type	Description
type	enum: Expression	Expression type.
value	string	Expression value.

FailActivity

This activity will fail within its own scope and output a custom error message and error code. The error message and code can provided either as a string literal or as an expression that can be evaluated to a string at runtime. The activity scope can be the whole pipeline or a control activity (e.g. foreach, switch, until), if the fail activity is contained in it.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Fail	Type of activity.
typeProperties.errorCode	object	The error code that categorizes the error type of the Fail activity. It can be dynamic content that's evaluated to a non empty/blank string at runtime. Type: string (or Expression with resultType string).
typeProperties.message	object	The error message that surfaced in the Fail activity. It can be dynamic content that's evaluated to a non empty/blank string at runtime. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

FilterActivity

Filter and return results from input array based on the conditions.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Filter	Type of activity.
typeProperties.condition	Expression	Condition to be used for filtering the input.
typeProperties.items	Expression	Input array on which filter should be applied.
userProperties	UserProperty[]	Activity user properties.

Folder

The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.

Name	Type	Description
name	string	The name of the folder that this Pipeline is in.

ForEachActivity

This activity is used for iterating over a collection and execute given activities.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: ForEach	Type of activity.
typeProperties.activities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities to execute .
typeProperties.batchCount	integer	Batch count to be used for controlling the number of parallel execution (when isSequential is set to false).
typeProperties.isSequential	boolean	Should the loop be executed in sequence or in parallel (max 50)
typeProperties.items	Expression	Collection to iterate.
userProperties	UserProperty[]	Activity user properties.

GetMetadataActivity

Activity to get metadata of dataset

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: GetMetadata	Type of activity.
typeProperties.dataset	DatasetReference	GetMetadata activity dataset reference.
typeProperties.fieldList	object[]	Fields of metadata to get from dataset.
typeProperties.formatSettings	FormatReadSettings	GetMetadata activity format settings.
typeProperties.storeSettings	StoreReadSettings	GetMetadata activity store settings.
userProperties	UserProperty[]	Activity user properties.

HDInsightActivityDebugInfoOption

The HDInsightActivityDebugInfoOption settings to use.

Name	Type	Description
Always	string
Failure	string
None	string

HDInsightHiveActivity

HDInsight Hive activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: HDInsightHive	Type of activity.
typeProperties.arguments	object[]	User specified arguments to HDInsightActivity.
typeProperties.defines	object	Allows user to specify defines for Hive job request.
typeProperties.getDebugInfo	HDInsightActivityDebugInfoOption	Debug info option.
typeProperties.queryTimeout	integer	Query timeout value (in minutes). Effective when the HDInsight cluster is with ESP (Enterprise Security Package)
typeProperties.scriptLinkedService	LinkedServiceReference	Script linked service reference.
typeProperties.scriptPath	object	Script path. Type: string (or Expression with resultType string).
typeProperties.storageLinkedServices	LinkedServiceReference[]	Storage linked service references.
typeProperties.variables	object	User specified arguments under hivevar namespace.
userProperties	UserProperty[]	Activity user properties.

HDInsightMapReduceActivity

HDInsight MapReduce activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: HDInsightMapReduce	Type of activity.
typeProperties.arguments	object[]	User specified arguments to HDInsightActivity.
typeProperties.className	object	Class name. Type: string (or Expression with resultType string).
typeProperties.defines	object	Allows user to specify defines for the MapReduce job request.
typeProperties.getDebugInfo	HDInsightActivityDebugInfoOption	Debug info option.
typeProperties.jarFilePath	object	Jar path. Type: string (or Expression with resultType string).
typeProperties.jarLibs	object[]	Jar libs.
typeProperties.jarLinkedService	LinkedServiceReference	Jar linked service reference.
typeProperties.storageLinkedServices	LinkedServiceReference[]	Storage linked service references.
userProperties	UserProperty[]	Activity user properties.

HDInsightPigActivity

HDInsight Pig activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: HDInsightPig	Type of activity.
typeProperties.arguments	object	User specified arguments to HDInsightActivity. Type: array (or Expression with resultType array).
typeProperties.defines	object	Allows user to specify defines for Pig job request.
typeProperties.getDebugInfo	HDInsightActivityDebugInfoOption	Debug info option.
typeProperties.scriptLinkedService	LinkedServiceReference	Script linked service reference.
typeProperties.scriptPath	object	Script path. Type: string (or Expression with resultType string).
typeProperties.storageLinkedServices	LinkedServiceReference[]	Storage linked service references.
userProperties	UserProperty[]	Activity user properties.

HDInsightSparkActivity

HDInsight Spark activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: HDInsightSpark	Type of activity.
typeProperties.arguments	object[]	The user-specified arguments to HDInsightSparkActivity.
typeProperties.className	string	The application's Java/Spark main class.
typeProperties.entryFilePath	object	The relative path to the root folder of the code/package to be executed. Type: string (or Expression with resultType string).
typeProperties.getDebugInfo	HDInsightActivityDebugInfoOption	Debug info option.
typeProperties.proxyUser	object	The user to impersonate that will execute the job. Type: string (or Expression with resultType string).
typeProperties.rootPath	object	The root path in 'sparkJobLinkedService' for all the job’s files. Type: string (or Expression with resultType string).
typeProperties.sparkConfig	object	Spark configuration property.
typeProperties.sparkJobLinkedService	LinkedServiceReference	The storage linked service for uploading the entry file and dependencies, and for receiving logs.
userProperties	UserProperty[]	Activity user properties.

HDInsightStreamingActivity

HDInsight streaming activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: HDInsightStreaming	Type of activity.
typeProperties.arguments	object[]	User specified arguments to HDInsightActivity.
typeProperties.combiner	object	Combiner executable name. Type: string (or Expression with resultType string).
typeProperties.commandEnvironment	object[]	Command line environment values.
typeProperties.defines	object	Allows user to specify defines for streaming job request.
typeProperties.fileLinkedService	LinkedServiceReference	Linked service reference where the files are located.
typeProperties.filePaths	object[]	Paths to streaming job files. Can be directories.
typeProperties.getDebugInfo	HDInsightActivityDebugInfoOption	Debug info option.
typeProperties.input	object	Input blob path. Type: string (or Expression with resultType string).
typeProperties.mapper	object	Mapper executable name. Type: string (or Expression with resultType string).
typeProperties.output	object	Output blob path. Type: string (or Expression with resultType string).
typeProperties.reducer	object	Reducer executable name. Type: string (or Expression with resultType string).
typeProperties.storageLinkedServices	LinkedServiceReference[]	Storage linked service references.
userProperties	UserProperty[]	Activity user properties.

IfConditionActivity

This activity evaluates a boolean expression and executes either the activities under the ifTrueActivities property or the ifFalseActivities property depending on the result of the expression.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: IfCondition	Type of activity.
typeProperties.expression	Expression	An expression that would evaluate to Boolean. This is used to determine the block of activities (ifTrueActivities or ifFalseActivities) that will be executed.
typeProperties.ifFalseActivities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities to execute if expression is evaluated to false. This is an optional property and if not provided, the activity will exit without any action.
typeProperties.ifTrueActivities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities to execute if expression is evaluated to true. This is an optional property and if not provided, the activity will exit without any action.
userProperties	UserProperty[]	Activity user properties.

IntegrationRuntimeReference

Integration runtime reference type.

Name	Type	Description
parameters	object	Arguments for integration runtime.
referenceName	string	Reference integration runtime name.
type	enum: IntegrationRuntimeReference	Type of integration runtime.

LinkedServiceReference

Linked service reference type.

Name	Type	Description
parameters	object	Arguments for LinkedService.
referenceName	string	Reference LinkedService name.
type	Type	Linked service reference type.

LogLocationSettings

Log location settings.

Name	Type	Description
linkedServiceName	LinkedServiceReference	Log storage linked service reference.
path	object	The path to storage for storing detailed logs of activity execution. Type: string (or Expression with resultType string).

LogSettings

Log settings.

Name	Type	Description
copyActivityLogSettings	CopyActivityLogSettings	Specifies settings for copy activity log.
enableCopyActivityLog	object	Specifies whether to enable copy activity log. Type: boolean (or Expression with resultType boolean).
logLocationSettings	LogLocationSettings	Log location settings customer needs to provide when enabling log.

LogStorageSettings

(Deprecated. Please use LogSettings) Log storage settings.

Name	Type	Description
enableReliableLogging	object	Specifies whether to enable reliable logging. Type: boolean (or Expression with resultType boolean).
linkedServiceName	LinkedServiceReference	Log storage linked service reference.
logLevel	object	Gets or sets the log level, support: Info, Warning. Type: string (or Expression with resultType string).
path	object	The path to storage for storing detailed logs of activity execution. Type: string (or Expression with resultType string).

LookupActivity

Lookup activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Lookup	Type of activity.
typeProperties.dataset	DatasetReference	Lookup activity dataset reference.
typeProperties.firstRowOnly	object	Whether to return first row or all rows. Default value is true. Type: boolean (or Expression with resultType boolean).
typeProperties.source	CopySource	Dataset-specific source properties, same as copy activity source.
userProperties	UserProperty[]	Activity user properties.

NotebookParameter

Notebook parameter.

Name	Type	Description
type	NotebookParameterType	Notebook parameter type.
value	object	Notebook parameter value. Type: string (or Expression with resultType string).

NotebookParameterType

Notebook parameter type.

Name	Type	Description
bool	string
float	string
int	string
string	string

NotebookReferenceType

Synapse notebook reference type.

Name	Type	Description
NotebookReference	string

ParameterSpecification

Definition of a single parameter for an entity.

Name	Type	Description
defaultValue	object	Default value of parameter.
type	ParameterType	Parameter type.

ParameterType

Parameter type.

Name	Type	Description
Array	string
Bool	string
Float	string
Int	string
Object	string
SecureString	string
String	string

PipelineElapsedTimeMetricPolicy

Pipeline ElapsedTime Metric Policy.

Name	Type	Description
duration	object	TimeSpan value, after which an Azure Monitoring Metric is fired.

PipelinePolicy

Pipeline Policy.

Name	Type	Description
elapsedTimeMetric	PipelineElapsedTimeMetricPolicy	Pipeline ElapsedTime Metric Policy.

PipelineReference

Pipeline reference type.

Name	Type	Description
name	string	Reference name.
referenceName	string	Reference pipeline name.
type	enum: PipelineReference	Pipeline reference type.

PipelineResource

Pipeline resource type.

Name	Type	Description
etag	string	Etag identifies change in the resource.
id	string	The resource identifier.
name	string	The resource name.
properties.activities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities in pipeline.
properties.annotations	object[]	List of tags that can be used for describing the Pipeline.
properties.concurrency	integer	The max number of concurrent runs for the pipeline.
properties.description	string	The description of the pipeline.
properties.folder	Folder	The folder that this Pipeline is in. If not specified, Pipeline will appear at the root level.
properties.parameters	<string,  ParameterSpecification>	List of parameters for pipeline.
properties.policy	PipelinePolicy	Pipeline Policy.
properties.runDimensions	object	Dimensions emitted by Pipeline.
properties.variables	<string,  VariableSpecification>	List of variables for pipeline.
type	string	The resource type.

PowerQuerySink

Power query sink.

Name	Type	Description
dataset	DatasetReference	Dataset reference.
description	string	Transformation description.
flowlet	DataFlowReference	Flowlet Reference
linkedService	LinkedServiceReference	Linked service reference.
name	string	Transformation name.
rejectedDataLinkedService	LinkedServiceReference	Rejected data linked service reference.
schemaLinkedService	LinkedServiceReference	Schema linked service reference.
script	string	sink script.

PowerQuerySinkMapping

Map Power Query mashup query to sink dataset(s).

Name	Type	Description
dataflowSinks	PowerQuerySink[]	List of sinks mapped to Power Query mashup query.
queryName	string	Name of the query in Power Query mashup document.

RedirectIncompatibleRowSettings

Redirect incompatible row settings

Name	Type	Description
linkedServiceName	object	Name of the Azure Storage, Storage SAS, or Azure Data Lake Store linked service used for redirecting incompatible row. Must be specified if redirectIncompatibleRowSettings is specified. Type: string (or Expression with resultType string).
path	object	The path for storing the redirect incompatible row data. Type: string (or Expression with resultType string).

ScriptActivity

Script activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Script	Type of activity.
typeProperties.logSettings	ScriptActivityTypeProperties.LogSettings	Log settings of script activity.
typeProperties.scriptBlockExecutionTimeout	object	ScriptBlock execution timeout. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).
typeProperties.scripts	ScriptActivityScriptBlock[]	Array of script blocks. Type: array.
userProperties	UserProperty[]	Activity user properties.

ScriptActivityLogDestination

The destination of logs. Type: string.

Name	Type	Description
ActivityOutput	string
ExternalStore	string

ScriptActivityParameter

Parameters of a script block.

Name	Type	Description
direction	ScriptActivityParameterDirection	The direction of the parameter.
name	object	The name of the parameter. Type: string (or Expression with resultType string).
size	integer	The size of the output direction parameter.
type	ScriptActivityParameterType	The type of the parameter.
value	object	The value of the parameter. Type: string (or Expression with resultType string).

ScriptActivityParameterDirection

The direction of the parameter.

Name	Type	Description
Input	string
InputOutput	string
Output	string

ScriptActivityParameterType

The type of the parameter.

Name	Type	Description
Boolean	string
DateTime	string
DateTimeOffset	string
Decimal	string
Double	string
Guid	string
Int16	string
Int32	string
Int64	string
Single	string
String	string
Timespan	string

ScriptActivityScriptBlock

Script block of scripts.

Name	Type	Description
parameters	ScriptActivityParameter[]	Array of script parameters. Type: array.
text	object	The query text. Type: string (or Expression with resultType string).
type	object	The type of the query. Please refer to the ScriptType for valid options. Type: string (or Expression with resultType string).

ScriptActivityTypeProperties.LogSettings

Log settings of script activity.

Name	Type	Description
logDestination	ScriptActivityLogDestination	The destination of logs. Type: string.
logLocationSettings	LogLocationSettings	Log location settings customer needs to provide when enabling log.

SecureInputOutputPolicy

Execution policy for an activity that supports secure input and output.

Name	Type	Description
secureInput	boolean	When set to true, Input from activity is considered as secure and will not be logged to monitoring.
secureOutput	boolean	When set to true, Output from activity is considered as secure and will not be logged to monitoring.

SecureString

Azure Data Factory secure string definition. The string value will be masked with asterisks '*' during Get or List API calls.

Name	Type	Description
type	string: SecureString	Type of the secret.
value	string	Value of secure string.

SetVariableActivity

Set value for a Variable.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	SecureInputOutputPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: SetVariable	Type of activity.
typeProperties.setSystemVariable	boolean	If set to true, it sets the pipeline run return value.
typeProperties.value	object	Value to be set. Could be a static value or Expression.
typeProperties.variableName	string	Name of the variable whose value needs to be set.
userProperties	UserProperty[]	Activity user properties.

SkipErrorFile

Skip error file.

Name	Type	Description
dataInconsistency	object	Skip if source/sink file changed by other concurrent write. Default is false. Type: boolean (or Expression with resultType boolean).
fileMissing	object	Skip if file is deleted by other client during copy. Default is true. Type: boolean (or Expression with resultType boolean).

SparkConfigurationParametrizationReference

Spark configuration reference.

Name	Type	Description
referenceName	object	Reference spark configuration name. Type: string (or Expression with resultType string).
type	SparkConfigurationReferenceType	Spark configuration reference type.

SparkConfigurationReferenceType

Spark configuration reference type.

Name	Type	Description
SparkConfigurationReference	string

SparkJobReferenceType

Synapse spark job reference type.

Name	Type	Description
SparkJobDefinitionReference	string

SqlServerStoredProcedureActivity

SQL stored procedure activity type.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: SqlServerStoredProcedure	Type of activity.
typeProperties.storedProcedureName	object	Stored procedure name. Type: string (or Expression with resultType string).
typeProperties.storedProcedureParameters	object	Value and type setting for stored procedure parameters. Example: "{Parameter1: {value: "1", type: "int"}}".
userProperties	UserProperty[]	Activity user properties.

SSISAccessCredential

SSIS access credential.

Name	Type	Description
domain	object	Domain for windows authentication. Type: string (or Expression with resultType string).
password	SecretBase: AzureKeyVaultSecretReference SecureString	Password for windows authentication.
userName	object	UseName for windows authentication. Type: string (or Expression with resultType string).

SSISChildPackage

SSIS embedded child package.

Name	Type	Description
packageContent	object	Content for embedded child package. Type: string (or Expression with resultType string).
packageLastModifiedDate	string	Last modified date for embedded child package.
packageName	string	Name for embedded child package.
packagePath	object	Path for embedded child package. Type: string (or Expression with resultType string).

SSISExecutionCredential

SSIS package execution credential.

Name	Type	Description
domain	object	Domain for windows authentication. Type: string (or Expression with resultType string).
password	SecureString	Password for windows authentication.
userName	object	UseName for windows authentication. Type: string (or Expression with resultType string).

SSISExecutionParameter

SSIS execution parameter.

Name	Type	Description
value	object	SSIS package execution parameter value. Type: string (or Expression with resultType string).

SSISLogLocation

SSIS package execution log location

Name	Type	Description
logPath	object	The SSIS package execution log path. Type: string (or Expression with resultType string).
type	SsisLogLocationType	The type of SSIS log location.
typeProperties.accessCredential	SSISAccessCredential	The package execution log access credential.
typeProperties.logRefreshInterval	object	Specifies the interval to refresh log. The default interval is 5 minutes. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).

SsisLogLocationType

The type of SSIS log location.

Name	Type	Description
File	string

SSISPackageLocation

SSIS package location.

Name	Type	Description
packagePath	object	The SSIS package path. Type: string (or Expression with resultType string).
type	SsisPackageLocationType	The type of SSIS package location.
typeProperties.accessCredential	SSISAccessCredential	The package access credential.
typeProperties.childPackages	SSISChildPackage[]	The embedded child package list.
typeProperties.configurationAccessCredential	SSISAccessCredential	The configuration file access credential.
typeProperties.configurationPath	object	The configuration file of the package execution. Type: string (or Expression with resultType string).
typeProperties.packageContent	object	The embedded package content. Type: string (or Expression with resultType string).
typeProperties.packageLastModifiedDate	string	The embedded package last modified date.
typeProperties.packageName	string	The package name.
typeProperties.packagePassword	SecretBase: AzureKeyVaultSecretReference SecureString	Password of the package.

SsisPackageLocationType

The type of SSIS package location.

Name	Type	Description
File	string
InlinePackage	string
PackageStore	string
SSISDB	string

SSISPropertyOverride

SSIS property override.

Name	Type	Description
isSensitive	boolean	Whether SSIS package property override value is sensitive data. Value will be encrypted in SSISDB if it is true
value	object	SSIS package property override value. Type: string (or Expression with resultType string).

StagingSettings

Staging settings.

Name	Type	Description
enableCompression	object	Specifies whether to use compression when copying data via an interim staging. Default value is false. Type: boolean (or Expression with resultType boolean).
linkedServiceName	LinkedServiceReference	Staging linked service reference.
path	object	The path to storage for storing the interim data. Type: string (or Expression with resultType string).

SwitchActivity

This activity evaluates an expression and executes activities under the cases property that correspond to the expression evaluation expected in the equals property.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Switch	Type of activity.
typeProperties.cases	SwitchCase[]	List of cases that correspond to expected values of the 'on' property. This is an optional property and if not provided, the activity will execute activities provided in defaultActivities.
typeProperties.defaultActivities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities to execute if no case condition is satisfied. This is an optional property and if not provided, the activity will exit without any action.
typeProperties.on	Expression	An expression that would evaluate to a string or integer. This is used to determine the block of activities in cases that will be executed.
userProperties	UserProperty[]	Activity user properties.

SwitchCase

Switch cases with have a value and corresponding activities.

Name	Type	Description
activities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities to execute for satisfied case condition.
value	string	Expected value that satisfies the expression result of the 'on' property.

SynapseNotebookActivity

Execute Synapse notebook activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: SynapseNotebook	Type of activity.
typeProperties.conf	object	Spark configuration properties, which will override the 'conf' of the notebook you provide.
typeProperties.configurationType	ConfigurationType	The type of the spark config.
typeProperties.driverSize	object	Number of core and memory to be used for driver allocated in the specified Spark pool for the session, which will be used for overriding 'driverCores' and 'driverMemory' of the notebook you provide. Type: string (or Expression with resultType string).
typeProperties.executorSize	object	Number of core and memory to be used for executors allocated in the specified Spark pool for the session, which will be used for overriding 'executorCores' and 'executorMemory' of the notebook you provide. Type: string (or Expression with resultType string).
typeProperties.notebook	SynapseNotebookReference	Synapse notebook reference.
typeProperties.numExecutors	object	Number of executors to launch for this session, which will override the 'numExecutors' of the notebook you provide. Type: integer (or Expression with resultType integer).
typeProperties.parameters	<string,  NotebookParameter>	Notebook parameters.
typeProperties.sparkConfig	object	Spark configuration property.
typeProperties.sparkPool	BigDataPoolParametrizationReference	The name of the big data pool which will be used to execute the notebook.
typeProperties.targetSparkConfiguration	SparkConfigurationParametrizationReference	The spark configuration of the spark job.
userProperties	UserProperty[]	Activity user properties.

SynapseNotebookReference

Synapse notebook reference type.

Name	Type	Description
referenceName	object	Reference notebook name. Type: string (or Expression with resultType string).
type	NotebookReferenceType	Synapse notebook reference type.

SynapseSparkJobDefinitionActivity

Execute spark job activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: SparkJob	Type of activity.
typeProperties.args	SynapseSparkJobActivityTypeProperties.Args[]	User specified arguments to SynapseSparkJobDefinitionActivity.
typeProperties.className	object	The fully-qualified identifier or the main class that is in the main definition file, which will override the 'className' of the spark job definition you provide. Type: string (or Expression with resultType string).
typeProperties.conf	object	Spark configuration properties, which will override the 'conf' of the spark job definition you provide.
typeProperties.configurationType	ConfigurationType	The type of the spark config.
typeProperties.driverSize	object	Number of core and memory to be used for driver allocated in the specified Spark pool for the job, which will be used for overriding 'driverCores' and 'driverMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).
typeProperties.executorSize	object	Number of core and memory to be used for executors allocated in the specified Spark pool for the job, which will be used for overriding 'executorCores' and 'executorMemory' of the spark job definition you provide. Type: string (or Expression with resultType string).
typeProperties.file	object	The main file used for the job, which will override the 'file' of the spark job definition you provide. Type: string (or Expression with resultType string).
typeProperties.files	object[]	(Deprecated. Please use pythonCodeReference and filesV2) Additional files used for reference in the main definition file, which will override the 'files' of the spark job definition you provide.
typeProperties.filesV2	object[]	Additional files used for reference in the main definition file, which will override the 'jars' and 'files' of the spark job definition you provide.
typeProperties.numExecutors	object	Number of executors to launch for this job, which will override the 'numExecutors' of the spark job definition you provide. Type: integer (or Expression with resultType integer).
typeProperties.pythonCodeReference	object[]	Additional python code files used for reference in the main definition file, which will override the 'pyFiles' of the spark job definition you provide.
typeProperties.scanFolder	object	Scanning subfolders from the root folder of the main definition file, these files will be added as reference files. The folders named 'jars', 'pyFiles', 'files' or 'archives' will be scanned, and the folders name are case sensitive. Type: boolean (or Expression with resultType boolean).
typeProperties.sparkConfig	object	Spark configuration property.
typeProperties.sparkJob	SynapseSparkJobReference	Synapse spark job reference.
typeProperties.targetBigDataPool	BigDataPoolParametrizationReference	The name of the big data pool which will be used to execute the spark batch job, which will override the 'targetBigDataPool' of the spark job definition you provide.
typeProperties.targetSparkConfiguration	SparkConfigurationParametrizationReference	The spark configuration of the spark job.
userProperties	UserProperty[]	Activity user properties.

SynapseSparkJobReference

Synapse spark job reference type.

Name	Type	Description
referenceName	object	Reference spark job name. Expression with resultType string.
type	SparkJobReferenceType	Synapse spark job reference type.

Type

Linked service reference type.

Name	Type	Description
LinkedServiceReference	string

UntilActivity

This activity executes inner activities until the specified boolean expression results to true or timeout is reached, whichever is earlier.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Until	Type of activity.
typeProperties.activities	Activity[]: AppendVariableActivity[] AzureDataExplorerCommandActivity[] AzureFunctionActivity[] AzureMLBatchExecutionActivity[] AzureMLExecutePipelineActivity[] AzureMLUpdateResourceActivity[] ControlActivity[] CopyActivity[] CustomActivity[] DataLakeAnalyticsUSQLActivity[] DatabricksNotebookActivity[] DatabricksSparkJarActivity[] DatabricksSparkPythonActivity[] DeleteActivity[] ExecuteDataFlowActivity[] ExecutePipelineActivity[] ExecuteSSISPackageActivity[] ExecuteWranglingDataflowActivity[] ExecutionActivity[] FailActivity[] FilterActivity[] ForEachActivity[] GetMetadataActivity[] HDInsightHiveActivity[] HDInsightMapReduceActivity[] HDInsightPigActivity[] HDInsightSparkActivity[] HDInsightStreamingActivity[] IfConditionActivity[] LookupActivity[] ScriptActivity[] SetVariableActivity[] SqlServerStoredProcedureActivity[] SwitchActivity[] SynapseNotebookActivity[] SynapseSparkJobDefinitionActivity[] UntilActivity[] ValidationActivity[] WaitActivity[] WebActivity[] WebHookActivity[]	List of activities to execute.
typeProperties.expression	Expression	An expression that would evaluate to Boolean. The loop will continue until this expression evaluates to true
typeProperties.timeout	object	Specifies the timeout for the activity to run. If there is no value specified, it takes the value of TimeSpan.FromDays(7) which is 1 week as default. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).
userProperties	UserProperty[]	Activity user properties.

UserProperty

User property.

Name	Type	Description
name	string	User property name.
value	object	User property value. Type: string (or Expression with resultType string).

ValidationActivity

This activity verifies that an external resource exists.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Validation	Type of activity.
typeProperties.childItems	object	Can be used if dataset points to a folder. If set to true, the folder must have at least one file. If set to false, the folder must be empty. Type: boolean (or Expression with resultType boolean).
typeProperties.dataset	DatasetReference	Validation activity dataset reference.
typeProperties.minimumSize	object	Can be used if dataset points to a file. The file must be greater than or equal in size to the value specified. Type: integer (or Expression with resultType integer).
typeProperties.sleep	object	A delay in seconds between validation attempts. If no value is specified, 10 seconds will be used as the default. Type: integer (or Expression with resultType integer).
typeProperties.timeout	object	Specifies the timeout for the activity to run. If there is no value specified, it takes the value of TimeSpan.FromDays(7) which is 1 week as default. Type: string (or Expression with resultType string), pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).
userProperties	UserProperty[]	Activity user properties.

VariableSpecification

Definition of a single variable for a Pipeline.

Name	Type	Description
defaultValue	object	Default value of variable.
type	VariableType	Variable type.

VariableType

Variable type.

Name	Type	Description
Array	string
Bool	string
String	string

WaitActivity

This activity suspends pipeline execution for the specified interval.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: Wait	Type of activity.
typeProperties.waitTimeInSeconds	object	Duration in seconds. Type: integer (or Expression with resultType integer).
userProperties	UserProperty[]	Activity user properties.

WebActivity

Web activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
linkedServiceName	LinkedServiceReference	Linked service reference.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	ActivityPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: WebActivity	Type of activity.
typeProperties.authentication	WebActivityAuthentication	Authentication method used for calling the endpoint.
typeProperties.body	object	Represents the payload that will be sent to the endpoint. Required for POST/PUT method, not allowed for GET method Type: string (or Expression with resultType string).
typeProperties.connectVia	IntegrationRuntimeReference	The integration runtime reference.
typeProperties.datasets	DatasetReference[]	List of datasets passed to web endpoint.
typeProperties.disableCertValidation	boolean	When set to true, Certificate validation will be disabled.
typeProperties.headers	object	Represents the headers that will be sent to the request. For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. Type: string (or Expression with resultType string).
typeProperties.httpRequestTimeout	object	Timeout for the HTTP request to get a response. Format is in TimeSpan (hh:mm:ss). This value is the timeout to get a response, not the activity timeout. The default value is 00:01:00 (1 minute). The range is from 1 to 10 minutes
typeProperties.linkedServices	LinkedServiceReference[]	List of linked services passed to web endpoint.
typeProperties.method	WebActivityMethod	Rest API method for target endpoint.
typeProperties.turnOffAsync	boolean	Option to disable invoking HTTP GET on location given in response header of a HTTP 202 Response. If set true, it stops invoking HTTP GET on http location given in response header. If set false then continues to invoke HTTP GET call on location given in http response headers.
typeProperties.url	object	Web activity target endpoint and path. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

WebActivityAuthentication

Web activity authentication properties.

Name	Type	Description
credential	CredentialReference	The credential reference containing authentication information.
password	SecretBase: AzureKeyVaultSecretReference SecureString	Password for the PFX file or basic authentication / Secret when used for ServicePrincipal
pfx	SecretBase: AzureKeyVaultSecretReference SecureString	Base64-encoded contents of a PFX file or Certificate when used for ServicePrincipal
resource	object	Resource for which Azure Auth token will be requested when using MSI Authentication. Type: string (or Expression with resultType string).
type	string	Web activity authentication (Basic/ClientCertificate/MSI/ServicePrincipal)
userTenant	object	TenantId for which Azure Auth token will be requested when using ServicePrincipal Authentication. Type: string (or Expression with resultType string).
username	object	Web activity authentication user name for basic authentication or ClientID when used for ServicePrincipal. Type: string (or Expression with resultType string).

WebActivityMethod

The list of HTTP methods supported by a WebActivity.

Name	Type	Description
DELETE	string
GET	string
POST	string
PUT	string

WebHookActivity

WebHook activity.

Name	Type	Description
dependsOn	ActivityDependency[]	Activity depends on condition.
description	string	Activity description.
name	string	Activity name.
onInactiveMarkAs	ActivityOnInactiveMarkAs	Status result of the activity when the state is set to Inactive. This is an optional property and if not provided when the activity is inactive, the status will be Succeeded by default.
policy	SecureInputOutputPolicy	Activity policy.
state	ActivityState	Activity state. This is an optional property and if not provided, the state will be Active by default.
type	string: WebHook	Type of activity.
typeProperties.authentication	WebActivityAuthentication	Authentication method used for calling the endpoint.
typeProperties.body	object	Represents the payload that will be sent to the endpoint. Required for POST/PUT method, not allowed for GET method Type: string (or Expression with resultType string).
typeProperties.headers	object	Represents the headers that will be sent to the request. For example, to set the language and type on a request: "headers" : { "Accept-Language": "en-us", "Content-Type": "application/json" }. Type: string (or Expression with resultType string).
typeProperties.method	WebHookActivityMethod	Rest API method for target endpoint.
typeProperties.reportStatusOnCallBack	object	When set to true, statusCode, output and error in callback request body will be consumed by activity. The activity can be marked as failed by setting statusCode >= 400 in callback request. Default is false. Type: boolean (or Expression with resultType boolean).
typeProperties.timeout	string	The timeout within which the webhook should be called back. If there is no value specified, it defaults to 10 minutes. Type: string. Pattern: ((\d+).)?(\d\d):(60|([0-5][0-9])):(60|([0-5][0-9])).
typeProperties.url	object	WebHook activity target endpoint and path. Type: string (or Expression with resultType string).
userProperties	UserProperty[]	Activity user properties.

WebHookActivityMethod

The list of HTTP methods supported by a WebHook activity.

Name	Type	Description
POST	string