Can we trigger a synapse pipeline based on Event Grid system topic output

Jakeer Shaik 106

Hi Team,

Currently we have an event grid system topic which triggers an azure function to create views on synapse serverless pool.

Once the above process is done, we move the data from serverless to dedicated sql pool.

we would like to automate the serverless to dedicated sql pool process as and when the serverless event grid process is done.

Is there a way to read the status of event grid system topic and trigger the serverless to dedicated sql pool process.

Thanks,
Jakeer. Shaik

HimanshuSinha-msft 19,381 Reputation points Microsoft Employee

2022-07-06T21:07:54.63+00:00
Hello @Jakeer Shaik ,
Thanks for the question and using MS Q&A platform.
As we understand the ask here is how to trigger the moving of records from the serverless to dedicated server when the Event grid topic process is done , please do let us know if its not accurate.
I am not sure but just wanted to pointed out that we do have event based triggers whcih to me seems worth a look . Have you looked into this , I am sharing the link below.

https://learn.microsoft.com/en-us/azure/data-factory/how-to-create-event-trigger?tabs=data-factory

Please do let me if you have any queries.
Thanks
Himanshu

Please don't forget to click on or upvote button whenever the information provided helps you. Original posters help the community find answers faster by identifying the correct answer. Here is how

Want a reminder to come back and check responses? Here is how to subscribe to a notification

If you are interested in joining the VM program and help shape the future of Q&A: Here is how you can be part of Q&A Volunteer Moderators
Jakeer Shaik 106 Reputation points

2022-07-07T06:50:52.14+00:00

Hi @HimanshuSinha-msft ,

Thanks for the prompt response.

Your understanding is correct. we need to trigger the serverless to dedicated server data processing once the D365 to serverless is done.

However, the link which you shared is pointing to storage event based triggers which is not suitable in my case.

Currently, our process goes like below

D365 --> Export to ADLS --> Event Grid System Topic --> Azure Functions --> Serverless pool

right after the above process is done, we want to start the below. We would want to automate the whole process.
Since there is no way for us to appraise once the serverless process is done, looking for a way to get this done.

Serverless pool --> dedicated sql pool

Let me know if this still creates confusion, I can provide more details.

Thanks,
Jakeer. Shaik
Jakeer Shaik 106 Reputation points

2022-07-12T06:29:27.81+00:00

@HimanshuSinha-msft , any inputs would be really appreciated.
HimanshuSinha-msft 19,381 Reputation points Microsoft Employee

2022-08-03T22:21:36.603+00:00

Hello @Jakeer Shaik ,
Apologies for the delay in reply from our side .Did you got a resolution for this ?
Just a quick thought since you are already using the Azure function , so I was thinking
Azure Functions --> Serverless pool - Azure Function -> Wrte to a conmtainer some dummy file - > We can create a event based trigger that this creation of the file will trigger a pipeline which should start the copy activity from serverless -> dediocated pool .
Does this sound logical ?

Thanks
Himanshu
Jakeer Shaik 106 Reputation points

2022-08-08T07:35:00.713+00:00

Hey @HimanshuSinha-msft

I did resolved this issue using event based triggers. How to close this?

Thanks,
Jakeer. Shaik

Share via

Can we trigger a synapse pipeline based on Event Grid system topic output