Batchpool - How to use job Manager task to "cleanup" after all tasks are completed?

SR-7559 141

What's the best way to perform a one time activity (of say, sending an email) when a job is complete?

My understanding is that job release task runs once on each node where atleast 1 task was run (according to https://learn.microsoft.com/en-us/azure/batch/batch-job-prep-release#job-release-task) and hence I'd not be able to use it.

What about Job Manager task? It certainly performs a 1 time 'init' activity (I can use it to submit all other regular tasks to the job). Thought I can use the same job manager task to run a 1-time cleanup activity per job? If not, what other option(s) do I have?

Thanks

prmanhas-MSFT 17,901 Reputation points Microsoft Employee

2021-07-16T04:53:40.47+00:00

@SR-7559 Thank you for your query!!!

Can you help me understand your end goal like what exactly is the kind of behavior you are expecting since I am little confused with the ask here

Once I have a better understanding I can check on same and will get back to you with my findings.

Thank you for your patience over the matter :)
SR-7559 141 Reputation points

2021-07-16T15:00:37.373+00:00

@prmanhas-MSFT ,

I'd like to be able to send an email (as an example) to the user that submitted the job once the job finishes on the pool. As you can imagine, this would be a 'one time' activity.

If I have a job release task, it will send as many emails as there are nodes according to https://learn.microsoft.com/en-us/azure/batch/batch-job-prep-release#job-release-task

My client application will be closed soon after the job is submitted (since the jobs are long running). and creating a new 'polling' feature is not optimal at all.

Edit: Task dependencies may be an option. But it's impractical to configure a task to run before '00000s of others are completed.

Hope this makes sense?
prmanhas-MSFT 17,901 Reputation points Microsoft Employee

2021-07-19T06:45:37.317+00:00

@SR-7559 Thank you for confirming on same and for giving detailed explanation on the mentioned ask.

I have already reached to our internal team on the mentioned ask to get better inputs and will keep you posted on the update.

Thank you for your patience over the matter :)

Accepted answer

prmanhas-MSFT 17,901 Reputation points Microsoft Employee

2021-07-20T16:27:54.027+00:00

@SR-7559 Thank you for your patience over the matter!!!

I had discussion internally and got below inputs from our internal team:

I believe the options are:

JobRelease which will have the effect of multiple emails as mentioned by you.
TaskDependencies which is the simplest way with a task the depends on all the others.
Have a JobManager task(or some other resource) poll the job and do something once complete.

Hope it helps!!!

Please "Accept as Answer" if it helped so it can help others in community looking for help on similar topics.
Please sign in to rate this answer.
SR-7559 141 Reputation points

2021-07-20T18:08:35.327+00:00

TThanks - Will have to use the task dependency route then. Not the greatest solution, but this would probably work for me - https://learn.microsoft.com/en-us/dotnet/api/microsoft.azure.batch.taskdependencies.onidrange?view=azure-dotnet

prmanhas-MSFT 17,901 Reputation points Microsoft Employee

2021-07-22T05:03:18.913+00:00

Yes @SR-7559 It seems to be the best solution as such for the use case.

Thanks
Sign in to comment

Use comments to ask for clarification, additional information, or improvements to the question.

Share via

Batchpool - How to use job Manager task to "cleanup" after all tasks are completed?

0 additional answers

Your answer