Редактиране

Споделяне чрез

Validation activity in Azure Data Factory and Synapse Analytics pipelines

  • Статия

APPLIES TO: Azure Data Factory Azure Synapse Analytics

Tip

Try out Data Factory in Microsoft Fabric, an all-in-one analytics solution for enterprises. Microsoft Fabric covers everything from data movement to data science, real-time analytics, business intelligence, and reporting. Learn how to start a new trial for free!

You can use a Validation in a pipeline to ensure the pipeline only continues execution once it validates the attached dataset reference exists, that it meets the specified criteria, or times out.

Create a Validation activity with UI

To use a Validation activity in a pipeline, complete the following steps:

  1. Search for Validation in the pipeline Activities pane, and drag a Validation activity to the pipeline canvas.
  2. Select the new Validation activity on the canvas if it isn't already selected, and its Settings tab, to edit its details. Screenshot shows the UI for a Validation activity.
  3. Select a dataset, or define a new one by selecting the New button. For file based datasets like the delimited text example above, you can select either a specific file, or a folder. When a folder is selected, the Validation activity allows you to ignore validation of the existence of child items in the folder, or require whether child items exist or not.
  4. The output of the Validation activity can be used as an input to any other activities, and referenced within those activities for any of their properties using dynamic expressions.

Syntax


{
"name": "Validation_Activity",
"type": "Validation",
"typeProperties": {
"dataset": {
"referenceName": "Storage_File",
"type": "DatasetReference"
},
"timeout": "0.12:00:00",
"sleep": 10,
"minimumSize": 20
}
},
{
"name": "Validation_Activity_Folder",
"type": "Validation",
"typeProperties": {
"dataset": {
"referenceName": "Storage_Folder",
"type": "DatasetReference"
},
"timeout": "0.12:00:00",
"sleep": 10,
"childItems": true
}
}

Type properties

Property Description Allowed values Required
name Name of the 'Validation' activity String Yes
type Must be set to Validation. String Yes
dataset Activity blocks execution until it has validated this dataset reference exists and that it meets the specified criteria, or timeout has been reached. Dataset provided should support "MinimumSize" or "ChildItems" property. Dataset reference Yes
timeout Specifies the timeout for the activity to run. If no value is specified, default value is 12 hours ("0.12:00:00"). Format is d.hh:mm:ss String No
sleep A delay in seconds between validation attempts. If no value is specified, default value is 10 seconds. Integer No
childItems Checks if the folder has child items.
true: Validates the folder exists and has items. Blocks until at least one item is present in the folder or timeout value is reached.
false: Validates the folder exists and is empty. Blocks until folder is empty or until timeout value is reached. If no value is specified, activity will block until the folder exists or until timeout is reached.		 Boolean No
minimumSize Minimum size of a file in bytes. If no value is specified, default value is 0 bytes Integer No

Related content

See other supported control flow activities: