Data Flows - Get

Service:: Data Factory

API Version:: 2018-06-01

Gets a data flow.

GET https://management.azure.com/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/Microsoft.DataFactory/factories/{factoryName}/dataflows/{dataFlowName}?api-version=2018-06-01

URI Parameters

Name	In	Required	Type	Description
dataFlowName	path	True	string minLength: 1 maxLength: 260 pattern: ^[A-Za-z0-9_][^<>#.%&:\\+?/]$	The data flow name.
factoryName	path	True	string minLength: 3 maxLength: 63 pattern: ^[A-Za-z0-9]+(?:-[A-Za-z0-9]+)*$	The factory name.
resourceGroupName	path	True	string minLength: 1 maxLength: 90	The name of the resource group. The name is case insensitive.
subscriptionId	path	True	string (uuid)	The ID of the target subscription. The value must be an UUID.
api-version	query	True	string minLength: 1	The API version to use for this operation.

Request Header

Name	Required	Type	Description
if-none-match		string	ETag of the data flow entity. Should only be specified for get. If the ETag matches the existing entity tag, or if * was provided, then no content will be returned.

Responses

Name	Type	Description
200 OK	DataFlowResource	Azure operation completed successfully.
Other Status Codes	CloudError	An unexpected error response.

Security

azure_auth

Azure Active Directory OAuth2 Flow.

Type: oauth2
Flow: implicit
Authorization URL: https://login.microsoftonline.com/common/oauth2/authorize

Scopes

Name	Description
user_impersonation	impersonate your user account

Examples

DataFlows_Get

Sample request

GET https://management.azure.com/subscriptions/12345678-1234-1234-1234-123456789012/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/dataflows/exampleDataFlow?api-version=2018-06-01

from azure.identity import DefaultAzureCredential

from azure.mgmt.datafactory import DataFactoryManagementClient

"""
# PREREQUISITES
    pip install azure-identity
    pip install azure-mgmt-datafactory
# USAGE
    python data_flows_get.py

    Before run the sample, please set the values of the client ID, tenant ID and client secret
    of the AAD application as environment variables: AZURE_CLIENT_ID, AZURE_TENANT_ID,
    AZURE_CLIENT_SECRET. For more info about how to get the value, please see:
    https://docs.microsoft.com/azure/active-directory/develop/howto-create-service-principal-portal
"""


def main():
    client = DataFactoryManagementClient(
        credential=DefaultAzureCredential(),
        subscription_id="12345678-1234-1234-1234-12345678abc",
    )

    response = client.data_flows.get(
        resource_group_name="exampleResourceGroup",
        factory_name="exampleFactoryName",
        data_flow_name="exampleDataFlow",
    )
    print(response)


# x-ms-original-file: specification/datafactory/resource-manager/Microsoft.DataFactory/DataFactory/stable/2018-06-01/examples/DataFlows_Get.json
if __name__ == "__main__":
    main()

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

using Azure;
using Azure.ResourceManager;
using System;
using System.Threading.Tasks;
using Azure.Core;
using Azure.Identity;
using Azure.ResourceManager.DataFactory.Models;
using Azure.ResourceManager.DataFactory;

// Generated from example definition: specification/datafactory/resource-manager/Microsoft.DataFactory/DataFactory/stable/2018-06-01/examples/DataFlows_Get.json
// this example is just showing the usage of "DataFlows_Get" operation, for the dependent resources, they will have to be created separately.

// get your azure access token, for more details of how Azure SDK get your access token, please refer to https://learn.microsoft.com/en-us/dotnet/azure/sdk/authentication?tabs=command-line
TokenCredential cred = new DefaultAzureCredential();
// authenticate your client
ArmClient client = new ArmClient(cred);

// this example assumes you already have this DataFactoryResource created on azure
// for more information of creating DataFactoryResource, please refer to the document of DataFactoryResource
string subscriptionId = "12345678-1234-1234-1234-12345678abc";
string resourceGroupName = "exampleResourceGroup";
string factoryName = "exampleFactoryName";
ResourceIdentifier dataFactoryResourceId = DataFactoryResource.CreateResourceIdentifier(subscriptionId, resourceGroupName, factoryName);
DataFactoryResource dataFactory = client.GetDataFactoryResource(dataFactoryResourceId);

// get the collection of this DataFactoryDataFlowResource
DataFactoryDataFlowCollection collection = dataFactory.GetDataFactoryDataFlows();

// invoke the operation
string dataFlowName = "exampleDataFlow";
NullableResponse<DataFactoryDataFlowResource> response = await collection.GetIfExistsAsync(dataFlowName);
DataFactoryDataFlowResource result = response.HasValue ? response.Value : null;

if (result == null)
{
    Console.WriteLine("Succeeded with null as result");
}
else
{
    // the variable result is a resource, you could call other operations on this instance as well
    // but just for demo, we get its data from this resource instance
    DataFactoryDataFlowData resourceData = result.Data;
    // for demo we just print out the id
    Console.WriteLine($"Succeeded on id: {resourceData.Id}");
}

To use the Azure SDK library in your project, see this documentation. To provide feedback on this code sample, open a GitHub issue

Sample response

Status code:: 200

Date: Mon, 22 Apr 2019 18:33:50 GMT
X-Content-Type-Options: nosniff
x-ms-correlation-request-id: 48983fb5-612a-44c6-8158-add7ffebb32b
x-ms-ratelimit-remaining-subscription-reads: 11993
x-ms-request-id: 48983fb5-612a-44c6-8158-add7ffebb32b

{
  "name": "exampleDataFlow",
  "type": "Microsoft.DataFactory/factories/dataflows",
  "etag": "15004c4f-0000-0200-0000-5cbe090e0000",
  "id": "/subscriptions/12345678-1234-1234-1234-123456789012/resourceGroups/exampleResourceGroup/providers/Microsoft.DataFactory/factories/exampleFactoryName/dataflows/exampleDataFlow",
  "properties": {
    "type": "MappingDataFlow",
    "description": "Sample demo data flow to convert currencies showing usage of union, derive and conditional split transformation.",
    "typeProperties": {
      "script": "source(output(PreviousConversionRate as double,Country as string,DateTime1 as string,CurrentConversionRate as double),allowSchemaDrift: false,validateSchema: false) ~> USDCurrency\nsource(output(PreviousConversionRate as double,Country as string,DateTime1 as string,CurrentConversionRate as double),allowSchemaDrift: true,validateSchema: false) ~> CADSource\nUSDCurrency, CADSource union(byName: true)~> Union\nUnion derive(NewCurrencyRate = round(CurrentConversionRate*1.25)) ~> NewCurrencyColumn\nNewCurrencyColumn split(Country == 'USD',Country == 'CAD',disjoint: false) ~> ConditionalSplit1@(USD, CAD)\nConditionalSplit1@USD sink(saveMode:'overwrite' ) ~> USDSink\nConditionalSplit1@CAD sink(saveMode:'overwrite' ) ~> CADSink",
      "sinks": [
        {
          "name": "USDSink",
          "dataset": {
            "type": "DatasetReference",
            "referenceName": "USDOutput"
          }
        },
        {
          "name": "CADSink",
          "dataset": {
            "type": "DatasetReference",
            "referenceName": "CADOutput"
          }
        }
      ],
      "sources": [
        {
          "name": "USDCurrency",
          "dataset": {
            "type": "DatasetReference",
            "referenceName": "CurrencyDatasetUSD"
          }
        },
        {
          "name": "CADSource",
          "dataset": {
            "type": "DatasetReference",
            "referenceName": "CurrencyDatasetCAD"
          }
        }
      ]
    }
  }
}

Definitions

Name	Description
CloudError	The object that defines the structure of an Azure Data Factory error response.
createdByType	The type of identity that created the resource.
DataFlowFolder	The folder that this data flow is in. If not specified, Data flow will appear at the root level.
DataFlowReference	Data flow reference type.
DataFlowReferenceType	Data flow reference type.
DataFlowResource	Data flow resource type.
DataFlowSink	Transformation for data flow sink.
DataFlowSource	Transformation for data flow source.
DatasetReference	Dataset reference type.
DatasetReferenceType	Dataset reference type.
Flowlet	Data flow flowlet
LinkedServiceReference	Linked service reference type.
MappingDataFlow	Mapping data flow.
PowerQuerySource	Power query source.
systemData	Metadata pertaining to creation and last modification of the resource.
Transformation	A data flow transformation.
Type	Linked service reference type.
WranglingDataFlow	Power Query data flow.

CloudError

Object

The object that defines the structure of an Azure Data Factory error response.

Name	Type	Description
error.code	string	Error code.
error.details	CloudError[]	Array with additional error details.
error.message	string	Error message.
error.target	string	Property name/path in request associated with error.

createdByType

Enumeration

The type of identity that created the resource.

Value	Description
User
Application
ManagedIdentity
Key

DataFlowFolder

Object

The folder that this data flow is in. If not specified, Data flow will appear at the root level.

Name	Type	Description
name	string	The name of the folder that this data flow is in.

DataFlowReference

Object

Data flow reference type.

Name	Type	Description
datasetParameters		Reference data flow parameters from dataset.
parameters		Data flow parameters
referenceName	string	Reference data flow name.
type	DataFlowReferenceType	Data flow reference type.

DataFlowReferenceType

Enumeration

Data flow reference type.

Value	Description
DataFlowReference	DataFlowReference

DataFlowResource

Object

Data flow resource type.

Name	Type	Description
etag	string	"If etag is provided in the response body, it may also be provided as a header per the normal etag convention. Entity tags are used for comparing two or more entities from the same requested resource. HTTP/1.1 uses entity tags in the etag (section 14.19), If-Match (section 14.24), If-None-Match (section 14.26), and If-Range (section 14.27) header fields.")
id	string (arm-id)	Fully qualified resource ID for the resource. E.g. "/subscriptions/{subscriptionId}/resourceGroups/{resourceGroupName}/providers/{resourceProviderNamespace}/{resourceType}/{resourceName}"
name	string	The name of the resource
properties	DataFlow: Flowlet MappingDataFlow WranglingDataFlow	Data flow properties.
systemData	systemData	Azure Resource Manager metadata containing createdBy and modifiedBy information.
type	string	The type of the resource. E.g. "Microsoft.Compute/virtualMachines" or "Microsoft.Storage/storageAccounts"

DataFlowSink

Object

Transformation for data flow sink.

Name	Type	Description
dataset	DatasetReference	Dataset reference.
description	string	Transformation description.
flowlet	DataFlowReference	Flowlet Reference
linkedService	LinkedServiceReference	Linked service reference.
name	string	Transformation name.
rejectedDataLinkedService	LinkedServiceReference	Rejected data linked service reference.
schemaLinkedService	LinkedServiceReference	Schema linked service reference.

DataFlowSource

Object

Transformation for data flow source.

Name	Type	Description
dataset	DatasetReference	Dataset reference.
description	string	Transformation description.
flowlet	DataFlowReference	Flowlet Reference
linkedService	LinkedServiceReference	Linked service reference.
name	string	Transformation name.
schemaLinkedService	LinkedServiceReference	Schema linked service reference.

DatasetReference

Object

Dataset reference type.

Name	Type	Description
parameters		Arguments for dataset.
referenceName	string	Reference dataset name.
type	DatasetReferenceType	Dataset reference type.

DatasetReferenceType

Enumeration

Dataset reference type.

Value	Description
DatasetReference	DatasetReference

Flowlet

Object

Data flow flowlet

Name	Type	Description
annotations	DataFlow.Annotations[]	List of tags that can be used for describing the data flow.
description	string	The description of the data flow.
folder	DataFlowFolder	The folder that this data flow is in. If not specified, Data flow will appear at the root level.
type	string: Flowlet	Type of data flow.
typeProperties.script	string	Flowlet script.
typeProperties.scriptLines	string[]	Flowlet script lines.
typeProperties.sinks	DataFlowSink[]	List of sinks in Flowlet.
typeProperties.sources	DataFlowSource[]	List of sources in Flowlet.
typeProperties.transformations	Transformation[]	List of transformations in Flowlet.

LinkedServiceReference

Object

Linked service reference type.

Name	Type	Description
parameters		Arguments for LinkedService.
referenceName	string	Reference LinkedService name.
type	Type	Linked service reference type.

MappingDataFlow

Object

Mapping data flow.

Name	Type	Description
annotations	DataFlow.Annotations[]	List of tags that can be used for describing the data flow.
description	string	The description of the data flow.
folder	DataFlowFolder	The folder that this data flow is in. If not specified, Data flow will appear at the root level.
type	string: MappingDataFlow	Type of data flow.
typeProperties.script	string	DataFlow script.
typeProperties.scriptLines	string[]	Data flow script lines.
typeProperties.sinks	DataFlowSink[]	List of sinks in data flow.
typeProperties.sources	DataFlowSource[]	List of sources in data flow.
typeProperties.transformations	Transformation[]	List of transformations in data flow.

PowerQuerySource

Object

Power query source.

Name	Type	Description
dataset	DatasetReference	Dataset reference.
description	string	Transformation description.
flowlet	DataFlowReference	Flowlet Reference
linkedService	LinkedServiceReference	Linked service reference.
name	string	Transformation name.
schemaLinkedService	LinkedServiceReference	Schema linked service reference.
script	string	source script.

systemData

Object

Metadata pertaining to creation and last modification of the resource.

Name	Type	Description
createdAt	string (date-time)	The timestamp of resource creation (UTC).
createdBy	string	The identity that created the resource.
createdByType	createdByType	The type of identity that created the resource.
lastModifiedAt	string (date-time)	The timestamp of resource last modification (UTC)
lastModifiedBy	string	The identity that last modified the resource.
lastModifiedByType	createdByType	The type of identity that last modified the resource.

Transformation

Object

A data flow transformation.

Name	Type	Description
dataset	DatasetReference	Dataset reference.
description	string	Transformation description.
flowlet	DataFlowReference	Flowlet Reference
linkedService	LinkedServiceReference	Linked service reference.
name	string	Transformation name.

Type

Enumeration

Linked service reference type.

Value	Description
LinkedServiceReference	LinkedServiceReference

WranglingDataFlow

Object

Power Query data flow.

Name	Type	Description
annotations	DataFlow.Annotations[]	List of tags that can be used for describing the data flow.
description	string	The description of the data flow.
folder	DataFlowFolder	The folder that this data flow is in. If not specified, Data flow will appear at the root level.
type	string: WranglingDataFlow	Type of data flow.
typeProperties.documentLocale	string	Locale of the Power query mashup document.
typeProperties.script	string	Power query mashup script.
typeProperties.sources	PowerQuerySource[]	List of sources in Power Query.

Share via

Data Flows - Get

URI Parameters

Request Header

Responses

Security

azure_auth

Scopes

Examples

DataFlows_Get

Sample request

Sample response

Definitions

CloudError

createdByType

DataFlowFolder

DataFlowReference

DataFlowReferenceType

DataFlowResource

DataFlowSink

DataFlowSource

DatasetReference

DatasetReferenceType

Flowlet

LinkedServiceReference

MappingDataFlow

PowerQuerySource

systemData

Transformation

Type

WranglingDataFlow