Episode
Ask the Expert: Smart Data Pipelines to Azure: Ingesting and migrating data the DataOps way
Ingesting and migrating data from a source system or legacy Hadoop stack to Azure platforms such as Synapse, ADLS and HDInsight may seem easy. But optimizing the data for a given Azure platform requires real data engineering. And operationalizing the data pipelines to run non-stop requires smart data pipelines that embed DataOps. Learn how StreamSets, a Microsoft startup program member, has helped customers speed their adoption of Azure cloud with its DataOps platform for smart data pipelines. Ready to learn more? https://streamsets.com and https://aka.ms/AzureMarketplaceStreamSets
Chapters
- 00:00 - Introduction
- 01:25 - What is StreamSets? What are some of the gaps you see in existing ETL tools?
- 05:13 - What are some of the biggest trends and issues with modern data applications?
- 08:56 - What's the breath of Azure technology supported by StreamSets? How to integrate with Azure Synapse?
- 11:51 - How do you see customers leveraging both these Azure Data Factory pipeline capabilities alongside StreamSets?
- 17:12 - What other Azure services or products does StreamSets agree with?
- 18:06 - How do you see DevOps and DataOps practices fitting into these data workflows and data pipelines? Why should developers care about these tools?
- 21:37 - How does StreamSets remains GDPR compliant?
- 24:39 - How quickly can StreamSets deploy a data pipeline? How is the deployment process?
- 26:37 - Does deploying workloads to Azure require rewrites?
- 27:10 - What are the typical target customers?
- 27:42 - Closing Notes
Ingesting and migrating data from a source system or legacy Hadoop stack to Azure platforms such as Synapse, ADLS and HDInsight may seem easy. But optimizing the data for a given Azure platform requires real data engineering. And operationalizing the data pipelines to run non-stop requires smart data pipelines that embed DataOps. Learn how StreamSets, a Microsoft startup program member, has helped customers speed their adoption of Azure cloud with its DataOps platform for smart data pipelines. Ready to learn more? https://streamsets.com and https://aka.ms/AzureMarketplaceStreamSets
Chapters
- 00:00 - Introduction
- 01:25 - What is StreamSets? What are some of the gaps you see in existing ETL tools?
- 05:13 - What are some of the biggest trends and issues with modern data applications?
- 08:56 - What's the breath of Azure technology supported by StreamSets? How to integrate with Azure Synapse?
- 11:51 - How do you see customers leveraging both these Azure Data Factory pipeline capabilities alongside StreamSets?
- 17:12 - What other Azure services or products does StreamSets agree with?
- 18:06 - How do you see DevOps and DataOps practices fitting into these data workflows and data pipelines? Why should developers care about these tools?
- 21:37 - How does StreamSets remains GDPR compliant?
- 24:39 - How quickly can StreamSets deploy a data pipeline? How is the deployment process?
- 26:37 - Does deploying workloads to Azure require rewrites?
- 27:10 - What are the typical target customers?
- 27:42 - Closing Notes
Have feedback? Submit an issue here.