Episode

Ask the Expert: Azure Data Factory

Ask the Expert

Have questions about data integration on Azure? The Azure Data Factory engineering teams are excited to hear from the community and learn how we can work together to build better products for you. Ready for more? Check out the documentation on Docs, or head to Learn.

Chapters

  • 00:00 - Introduction
  • 01:27 - Getting started with Azure Data Factory
  • 03:26 - What are your thoughts on data flows? How to scale and apply information to huge amounts of data?
  • 05:23 - Can you turn off Auto Update? If you want to make sure you have a stable version of self-hosted integration runtime, can you always rely on it? Or will you be forced to upgrade?
  • 07:23 - Does SHIR require an inbound port or just the outbound port?
  • 08:20 - Has the CI/CD experience with ADF been improved? Is it reliable now?
  • 09:34 - What's the rationale behind DIU in copy? How to understand its pricing?
  • 11:40 - What is TTL?
  • 13:23 - How do I optimize copying data?
  • 16:02 - What are event triggers?
  • 17:47 - What do you recommend to lock down the use of SHIR within Vnet?
  • 19:32 - How to optimize mapping data flow?
  • 22:31 - What is Azure Purview? How can you integrate it with ADF?
  • 24:31 - Can you build scalable and high availability solutions with self-hosted integration runtime? Is it supported?
  • 26:18 - What is the best place to start to learn about ADF? What's a good place for self-learners to get started on building pipelines and data flows?
  • 27:51 - What's the difference between ADF, Synapse pipelines, and data flows?
  • 29:09 - Closing Notes

Azure Data Factory