Data Factory Developer Reference

  • Article

 

Azure Data Factory is a cloud-based data integration service that orchestrates and automates the movement and transformation of data. Just like a manufacturing factory that runs equipment to take raw materials and transform them into finished goods, Data Factory orchestrates existing services that collect raw data and transform it into ready-to-use information.

Data Factory works across on-premises and cloud data sources and SaaS to ingest, prepare, transform, analyze, and publish your data. Use Data Factory to compose services into managed data flow pipelines to transform your data using services like Azure HDInsight (Hadoop) and Azure Batch for your big data computing needs, and with Azure Machine Learning to operationalize your analytics solutions. Go beyond just a tabular monitoring view, and use the rich visualizations of Data Factory to quickly display the lineage and dependencies between your data pipelines. Monitor all of your data flow pipelines from a single unified view to easily pinpoint issues and setup monitoring alerts.

See Azure Data Factory conceptual content for detailed conceptual information about the Azure Data Factory service. The MSDN Library contains the following reference content:

PowerShell Cmdlet Reference

You can use Azure PowerShell cmdlets to create, monitor, manage, and troubleshoot Azure data factories. See Data Factory Cmdlet Reference for detailed information about Azure Data Factory PowerShell cmdlets.

.NET Class Library Reference

You can programmatically create, monitor, and manage Azure data factories and create custom activities using Azure Data Factory .NET SDK. See Azure Data Factory Class Library Reference for comprehensive documentation of all classes in the class library.

See Use custom activities in Azure Data Factory pipeline and Create data factories programmatically using the .NET Class Library articles for walkthroughs.

REST API Reference

Azure Data Factory allows you to create, monitor, and manage data factories using the REST API. See Data Factory REST API Reference for details.