Custom event trigger on Azure Synapse pipeline

Salomon Henriquez 106 Reputation points Microsoft Employee
2022-11-01T16:52:13.32+00:00

– Does a custom event trigger can be triggered once a file has been dropped into external storage(file system) or destination (outside of an Azure storage account)?

Azure Synapse Analytics
Azure Synapse Analytics
An Azure analytics service that brings together data integration, enterprise data warehousing, and big data analytics. Previously known as Azure SQL Data Warehouse.
4,658 questions
0 comments No comments
{count} votes

Accepted answer
  1. ShaikMaheer-MSFT 38,326 Reputation points Microsoft Employee
    2022-11-02T09:33:39.407+00:00

    Hi @Salomon Henriquez ,

    Thank you for posting query in Microsoft Q&A Platform.

    Custom event triggers will actually run pipelines by subscribing to the events which are coming to a topic in Azure Event Grid. Click here to know more about Custom event triggers.

    In event grids world, Event sources will send events to topics which are inside event grid. Event handlers will subscribe to that event and whenever an event comes to topic, event handlers to trigger their processes.

    With above definition, it's clear that in case of Custom event triggers, Synapse Pipelines are Event Handlers. Because Synapse or ADF internally subscribe to topic in Azure Event grid and listens to events coming to topic and trigger pipelines accordingly.

    Now, coming to your query, in your case you can consider to having some process externally running which watches for file dropped cases in your external file system and send events to Even trigger topic. There by we need to subscribe that topic using Custom trigger to start your pipeline.

    But I feel best way is have a program or process running in your external environment which checks of file dropped cases and makes an API call to Synapse to run pipeline.

    Hope this helps. Please let me know if any further queries.

    --------------

    • Please don't forget to click on 130616-image.png or upvote 130671-image.png button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how
    • Want a reminder to come back and check responses? Here is how to subscribe to a notification

0 additional answers

Sort by: Most helpful