Continuous webjob cannot start after zipdeploy

Question

Continuous webjob cannot start after zipdeploy

Isaac Drew 0

We are having issues with our continuous webjobs after deploying them via zipdeploy.

Our flow for deployment is:

Use MSDeploy (msbuild) to publish our aspnetcore app to an azure app service
After that completes, deploy the web jobs via ZipDeploy.

After deployment, most of the time the continuous webjobs we deploy are in the "Stopped" state, and when we go to start them in the WebJob section of the app service, we get the following error: "Failed to run WebJob". Inspecting the API traffic, it seems that call is returning a 404...

It makes a POST to subscriptions/XXXX/resourceGroups/XXXX/providers/Microsoft.Web/sites/XXXX/continuouswebjobs/StewardshipActivity/start?api-version=2018-11-01, which returns a 404.

Doing az webapp webjob continuous start --name xxxx --resource-group xxxx --webjob-name StewardshipActivity also gives a 404.

This is the only log we're seeing within the jobs (3:15 is when the web app was deployed, so it's expected the jobs are stopped here.)

[10/23/2025 03:15:00 > fc69d9: SYS INFO] Status changed to Stopping
[10/23/2025 03:15:00 > fc69d9: INFO] info: Microsoft.Hosting.Lifetime[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Application is shutting down...
[10/23/2025 03:15:00 > fc69d9: INFO] info: Microsoft.Azure.WebJobs.Hosting.JobHostService[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Stopping JobHost
[10/23/2025 03:15:00 > fc69d9: INFO] info: Host.Startup[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Stopping the listener 'Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener' for function 'DailyActivity'
[10/23/2025 03:15:00 > fc69d9: INFO] info: Host.Startup[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Stopping the listener 'Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener' for function 'UpcomingActivity'
[10/23/2025 03:15:00 > fc69d9: INFO] info: Host.Startup[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Stopped the listener 'Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener' for function 'DailyActivity'
[10/23/2025 03:15:00 > fc69d9: INFO] info: Host.Startup[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Stopped the listener 'Microsoft.Azure.WebJobs.Host.Listeners.SingletonListener' for function 'UpcomingActivity'
[10/23/2025 03:15:00 > fc69d9: INFO] info: Host.Startup[0]
[10/23/2025 03:15:00 > fc69d9: INFO]       Job host stopped
[10/23/2025 03:15:00 > fc69d9: SYS INFO] Status changed to Success
[10/23/2025 03:15:00 > fc69d9: SYS INFO] Status changed to Stopped

If I go to Kudu, and go to C:\home\site\wwwroot\App_Data\jobs\continuous\StewardshipActivity, and run run.cmd, the job works without issue. So there seems to be a problem on the Azure side.

Anonymous

2025-10-23T09:31:17.4833333+00:00
Hi @Isaac Drew

Could you please verify that the folder structure is correct?

You can check this by running the following CLI command:

az webapp webjob continuous list --name <appName> --resource-group <rg>

If your WebJob does not appear in the output, it means Azure hasn’t properly registered it. This usually points to an incorrect folder structure or missing job metadata.
Isaac Drew 0 Reputation points

2025-10-24T01:01:55.31+00:00

@Anonymous This command showed the webjobs as expected, furthermore the webjobs were showing in the WebJob panel of the App Service the entire time... that's how we even got the error to begin with (clicking the Run button on the webjob in the portal).

We have now seemed to resolve it by forcing Kudu to restart (using the instructions at https://github.com/projectkudu/kudu/wiki/Full-stopping-a-Web-App ), however it seems this breaks again when we re-deploy.

We recently added creating an app_offline.htm file as part of our deploy, could this be causing any issues?
Anonymous

2025-10-24T05:48:44.95+00:00
Hi @Isaac Drew

Could you please confirm are you using ZipDeploy with Run-From-Package enabled?

How to Confirm Run-From-Package

Go to Kudu → Debug Console → D:\home\site\wwwroot.

If you see a single packagename.zip file and no expanded folders, you are in run-from-package mode.

Alternatively, check D:\home\data\SitePackages\—if there’s a zip file there, run-from-package is active.
Isaac Drew 0 Reputation points

2025-10-24T05:53:38.59+00:00

Hi @Anonymous

We are not using run-from-package mode. No zip files in either directory.
Anonymous

2025-10-24T06:27:13.9466667+00:00
Hi @Isaac Drew

Your WebJob itself is looks fine, but Azure Resource Manager (ARM) doesn’t have it registered, which is why the portal “Run” button and the az webapp webjob continuous start command fail with a 404.

If you want portal/CLI control, redeploy the WebJob using a method that updates ARM metadata like:

Visual Studio publish along with your main solution, or

ZipDeploy/MSDeploy via Azure CLI or Azure DevOps.

If you’re okay managing it via Kudu API or App Settings (WEBJOBS_STOPPED), you can skip redeployment.

could you try Visual Studio deploy along with your main solution for the simplest ARM integration ?
Isaac Drew 0 Reputation points

2025-10-24T06:51:05.6266667+00:00

Hi @Anonymous , is there any guidance on how to "Visual Studio publish along with your main solution"? This documentation page (https://learn.microsoft.com/en-us/azure/app-service/webjobs-dotnet-deploy-vs#webjobs-as-net-core-console-apps) suggests that ".NET Core Web Apps and/or .NET Core WebJobs can't be linked with web projects.".

Do you mean to MSDeploy each individual webjob, like in https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-get-started#deploy-as-a-webjob ?

One clarification here, this seems to only happen sometimes. Other times the webjobs start automatically after their deploy without issue.

This method of deploy has been working for us for over 1 year up until now, which makes it feel like something has changed internally in Azure/Kudu.
Anonymous

2025-10-24T07:44:26.84+00:00

Hi @Isaac Drew

The issue occurs because the Azure Portal and az webapp webjob commands rely on ARM’s WebJobs resource metadata, which sometimes lags behind the actual Kudu environment. After deployment (especially when using ZipDeploy or custom MSDeploy steps), ARM may not yet recognize the newly deployed continuous WebJob — leading to 404 errors when you try to start it through the portal or CLI.

To resolve this:

Restart the App Service after deployment to force ARM to refresh its WebJob metadata.

Alternatively, you can trigger the WebJob directly through Kudu (https://<yourapp>.scm.azurewebsites.net/), which reflects the real-time state.

Ensuring the deployment completes fully before starting the job (adding a short delay or health check) can also help.

This behavior is intermittent and due to propagation delay rather than an actual deployment failure.
Isaac Drew 0 Reputation points

2025-10-24T08:13:34.4233333+00:00
Hi @Anonymous .

The webjobs still fail even after a restart (unless we do a SCM restart too, https://github.com/projectkudu/kudu/wiki/Full-stopping-a-Web-App)

The 404 does not seem like a propogation delay, because this was after hours/days, not seconds.

We do not and should not need to ever manually start these continuous webjobs. The continuous webjobs not starting has not been an issue we've encountered, previously they would always restart as expected right after deploy. The only reason we've needed to go into the WebJob App Service panel and try manually starting it (causing the 404) is because they didn't automatically start for unknown reasons in these fail cases.

We're actually experiencing issues with our Triggered WebJobs as well. When the continuous webjobs fail, it seems the triggered webjobs with schedules also do not get rescheduled after deployment.

An excerpt from the job scheduler log for one of our triggered webjobs:

[10/21/2025 05:43:44 > fc69d9: SYS INFO] Scheduling WebJob with 0 0 2 * * * next schedule expected in 20:16:15.8949374 [10/22/2025 00:27:58 > fc69d9: SYS INFO] Removing current schedule from WebJob

What we would see is the above, Removing current schedule from WebJob would occur when we deploy (this is to be expected, as the webjob will be stopped when app deploys). However, the job does not get scheduled to run again afterwards.

I have a feeling this is correlated with the continuous not working.

In C:\home\LogFiles\kudu\trace in a different web app having the same issues, we see the following files:

The zipdeploy at 2025-10-24T05-08-26 had working webjobs after deploy. 2025-10-24T07-50-32 did not. The difference is there were no Shutdown and Startup events after the deploy where webjobs did not work. There aren't any differences in the two zipdeploy logs.
Brian Crowell 0 Reputation points

2025-10-27T03:58:53.75+00:00
I have been struggling with a similar situation with a continuous webjob only starting after every other deploy. The issue seems to be the lock and status files under /home/data/jobs, which aren't cleaned up properly during half of the cycle. I am using WEBSITE_RUN_FROM_PACKAGE with a single ZIP deployment.

Turning off is_singleton or setting a shutdown grace period for the continuous webjob did nothing to help. Here's what did:

Creating a symlink from /home/site/jobs to /home/site/wwwroot/jobs, where I am keeping my job definitions. For whatever reason, App Service did not like running my jobs directly out of wwwroot/App_Data/jobs. Copying the definitions to /home/site/jobs may work as well or better, but you might need to set up a separate deployment.

Setting is_in_place to false. The job is indeed copied to a directory under /tmp/jobs, which I would hope is writable.

Setting WEBSITE_DISABLE_OVERLAPPED_RECYCLING to "1". This made the biggest impact, and might be the only thing necessary. This seems to sidestep the issue with locks not being cleaned up properly.

I genuinely do not know what the best answer is. I do think this is definitely a bug in whatever manages the webjobs. The WebJobs page in the Azure website is not terribly helpful, because it seems to pick an instance at random to tell you the status of things. It also does not clear the "Completed" status on triggered jobs, nor tell you the last time the job ran. I thought the job had been every two minutes for weeks. It did not.

Best of luck. Hopefully Microsoft sends someone down with a wrench to fix this.
Anonymous

2025-10-28T01:40:23.9966667+00:00
Hi @Isaac Drew

This issue might occur due to an unhandled exception in the WebJob or because the deployment is being done in a single slot.

You can try the following steps to mitigate the issue:

Enable “Always On” under your Azure Web App → Configuration → General settings.

Deploy your application to a staging slot instead of directly to the production slot.

Reason:

If a WebJob encounters an unhandled exception or consumes excessive resources immediately after starting, the process may terminate. Azure’s process manager will attempt to restart it, but repeated failures can cause the job to remain permanently stopped.

When deploying to a single slot, the application restarts during deployment. At the same time, the WebJob runtime may attempt to start the job, which can result in a race condition — causing the job to fail and remain in a “Stopped” state.
Steve Tapley 5 Reputation points

2025-10-28T05:25:54.2766667+00:00
Hi Vimal,

This feels like such a low quality AI generated answer without a lot of thought behind it:

there are no unhandled exceptions happening. As explained, if we restart the SCM, then it works properly. Logging is turned on, and nothing is being reported.

"Always On" is already on (as you would know by looking at our instance)

We don't use staging slots

There are no repeated failures - restarting SCM once fixes the problem

If its a race condition, then its a bug inside the Azure platform

This has worked for well over a year - and now just stops working - leading us to believe that this is a recently introduced bug somewhere in the Azure platform

This is an intermittent problem - deploying the same code twice to Azure produces different results

As mentioned in Isaac's answer, we have worked around this bug by using a different publish method. The fact that a different publish method works strongly indicates there are no exceptions, server restarts, staging/production slot issues at play here - just a likely Azure platform issue.
Anonymous

2025-10-28T05:43:45.78+00:00

Hi @Isaac Drew / @Steve Tapley

Would you like me to escalate your question to next level if it is not helpful?
Steve Tapley 5 Reputation points

2025-10-28T08:37:42.7933333+00:00

Hi Vimal,

If you think its worthwhile fixing the bug, then please escalate. We will help you where we can.

Either way, our workaround solution seems to be working, so we will go with that.

1 answer

Your answer

Anonymous

2025-10-23T09:31:17.4833333+00:00

Hi @Isaac Drew

Could you please verify that the folder structure is correct?

You can check this by running the following CLI command:

az webapp webjob continuous list --name <appName> --resource-group <rg>

If your WebJob does not appear in the output, it means Azure hasn’t properly registered it. This usually points to an incorrect folder structure or missing job metadata.
Isaac Drew 0 Reputation points

2025-10-24T01:01:55.31+00:00

@Anonymous This command showed the webjobs as expected, furthermore the webjobs were showing in the WebJob panel of the App Service the entire time... that's how we even got the error to begin with (clicking the Run button on the webjob in the portal).

We have now seemed to resolve it by forcing Kudu to restart (using the instructions at https://github.com/projectkudu/kudu/wiki/Full-stopping-a-Web-App ), however it seems this breaks again when we re-deploy.

We recently added creating an app_offline.htm file as part of our deploy, could this be causing any issues?
Anonymous

2025-10-24T05:48:44.95+00:00

Hi @Isaac Drew

Could you please confirm are you using ZipDeploy with Run-From-Package enabled?

How to Confirm Run-From-Package

Go to Kudu → Debug Console → D:\home\site\wwwroot.

If you see a single packagename.zip file and no expanded folders, you are in run-from-package mode.

Alternatively, check D:\home\data\SitePackages\—if there’s a zip file there, run-from-package is active.
Isaac Drew 0 Reputation points

2025-10-24T05:53:38.59+00:00

Hi @Anonymous

We are not using run-from-package mode. No zip files in either directory.
Anonymous

2025-10-24T06:27:13.9466667+00:00

Hi @Isaac Drew

Your WebJob itself is looks fine, but Azure Resource Manager (ARM) doesn’t have it registered, which is why the portal “Run” button and the az webapp webjob continuous start command fail with a 404.

If you want portal/CLI control, redeploy the WebJob using a method that updates ARM metadata like:

Visual Studio publish along with your main solution, or

ZipDeploy/MSDeploy via Azure CLI or Azure DevOps.

If you’re okay managing it via Kudu API or App Settings (WEBJOBS_STOPPED), you can skip redeployment.

could you try Visual Studio deploy along with your main solution for the simplest ARM integration ?
Isaac Drew 0 Reputation points

2025-10-24T06:51:05.6266667+00:00

Hi @Anonymous , is there any guidance on how to "Visual Studio publish along with your main solution"? This documentation page (https://learn.microsoft.com/en-us/azure/app-service/webjobs-dotnet-deploy-vs#webjobs-as-net-core-console-apps) suggests that ".NET Core Web Apps and/or .NET Core WebJobs can't be linked with web projects.".

Do you mean to MSDeploy each individual webjob, like in https://learn.microsoft.com/en-us/azure/app-service/webjobs-sdk-get-started#deploy-as-a-webjob ?

One clarification here, this seems to only happen sometimes. Other times the webjobs start automatically after their deploy without issue.

This method of deploy has been working for us for over 1 year up until now, which makes it feel like something has changed internally in Azure/Kudu.
Anonymous

2025-10-24T07:44:26.84+00:00

Hi @Isaac Drew

The issue occurs because the Azure Portal and az webapp webjob commands rely on ARM’s WebJobs resource metadata, which sometimes lags behind the actual Kudu environment. After deployment (especially when using ZipDeploy or custom MSDeploy steps), ARM may not yet recognize the newly deployed continuous WebJob — leading to 404 errors when you try to start it through the portal or CLI.

To resolve this:

Restart the App Service after deployment to force ARM to refresh its WebJob metadata.

Alternatively, you can trigger the WebJob directly through Kudu (https://<yourapp>.scm.azurewebsites.net/), which reflects the real-time state.

Ensuring the deployment completes fully before starting the job (adding a short delay or health check) can also help.

This behavior is intermittent and due to propagation delay rather than an actual deployment failure.
Isaac Drew 0 Reputation points

2025-10-24T08:13:34.4233333+00:00

Hi @Anonymous .

The webjobs still fail even after a restart (unless we do a SCM restart too, https://github.com/projectkudu/kudu/wiki/Full-stopping-a-Web-App)

The 404 does not seem like a propogation delay, because this was after hours/days, not seconds.

We do not and should not need to ever manually start these continuous webjobs. The continuous webjobs not starting has not been an issue we've encountered, previously they would always restart as expected right after deploy. The only reason we've needed to go into the WebJob App Service panel and try manually starting it (causing the 404) is because they didn't automatically start for unknown reasons in these fail cases.

We're actually experiencing issues with our Triggered WebJobs as well. When the continuous webjobs fail, it seems the triggered webjobs with schedules also do not get rescheduled after deployment.

An excerpt from the job scheduler log for one of our triggered webjobs:

[10/21/2025 05:43:44 > fc69d9: SYS INFO] Scheduling WebJob with 0 0 2 * * * next schedule expected in 20:16:15.8949374 [10/22/2025 00:27:58 > fc69d9: SYS INFO] Removing current schedule from WebJob

What we would see is the above, Removing current schedule from WebJob would occur when we deploy (this is to be expected, as the webjob will be stopped when app deploys). However, the job does not get scheduled to run again afterwards.

I have a feeling this is correlated with the continuous not working.

In C:\home\LogFiles\kudu\trace in a different web app having the same issues, we see the following files:

The zipdeploy at 2025-10-24T05-08-26 had working webjobs after deploy. 2025-10-24T07-50-32 did not. The difference is there were no Shutdown and Startup events after the deploy where webjobs did not work. There aren't any differences in the two zipdeploy logs.
Brian Crowell 0 Reputation points

2025-10-27T03:58:53.75+00:00

I have been struggling with a similar situation with a continuous webjob only starting after every other deploy. The issue seems to be the lock and status files under /home/data/jobs, which aren't cleaned up properly during half of the cycle. I am using WEBSITE_RUN_FROM_PACKAGE with a single ZIP deployment.

Turning off is_singleton or setting a shutdown grace period for the continuous webjob did nothing to help. Here's what did:

Creating a symlink from /home/site/jobs to /home/site/wwwroot/jobs, where I am keeping my job definitions. For whatever reason, App Service did not like running my jobs directly out of wwwroot/App_Data/jobs. Copying the definitions to /home/site/jobs may work as well or better, but you might need to set up a separate deployment.

Setting is_in_place to false. The job is indeed copied to a directory under /tmp/jobs, which I would hope is writable.

Setting WEBSITE_DISABLE_OVERLAPPED_RECYCLING to "1". This made the biggest impact, and might be the only thing necessary. This seems to sidestep the issue with locks not being cleaned up properly.

I genuinely do not know what the best answer is. I do think this is definitely a bug in whatever manages the webjobs. The WebJobs page in the Azure website is not terribly helpful, because it seems to pick an instance at random to tell you the status of things. It also does not clear the "Completed" status on triggered jobs, nor tell you the last time the job ran. I thought the job had been every two minutes for weeks. It did not.

Best of luck. Hopefully Microsoft sends someone down with a wrench to fix this.
Anonymous

2025-10-28T01:40:23.9966667+00:00

Hi @Isaac Drew

This issue might occur due to an unhandled exception in the WebJob or because the deployment is being done in a single slot.

You can try the following steps to mitigate the issue:

Enable “Always On” under your Azure Web App → Configuration → General settings.

Deploy your application to a staging slot instead of directly to the production slot.

Reason:

If a WebJob encounters an unhandled exception or consumes excessive resources immediately after starting, the process may terminate. Azure’s process manager will attempt to restart it, but repeated failures can cause the job to remain permanently stopped.

When deploying to a single slot, the application restarts during deployment. At the same time, the WebJob runtime may attempt to start the job, which can result in a race condition — causing the job to fail and remain in a “Stopped” state.
Steve Tapley 5 Reputation points

2025-10-28T05:25:54.2766667+00:00

Hi Vimal,

This feels like such a low quality AI generated answer without a lot of thought behind it:

there are no unhandled exceptions happening. As explained, if we restart the SCM, then it works properly. Logging is turned on, and nothing is being reported.

"Always On" is already on (as you would know by looking at our instance)

We don't use staging slots

There are no repeated failures - restarting SCM once fixes the problem

If its a race condition, then its a bug inside the Azure platform

This has worked for well over a year - and now just stops working - leading us to believe that this is a recently introduced bug somewhere in the Azure platform

This is an intermittent problem - deploying the same code twice to Azure produces different results

As mentioned in Isaac's answer, we have worked around this bug by using a different publish method. The fact that a different publish method works strongly indicates there are no exceptions, server restarts, staging/production slot issues at play here - just a likely Azure platform issue.
Anonymous

2025-10-28T05:43:45.78+00:00

Hi @Isaac Drew / @Steve Tapley

Would you like me to escalate your question to next level if it is not helpful?
Steve Tapley 5 Reputation points

2025-10-28T08:37:42.7933333+00:00

Hi Vimal,

If you think its worthwhile fixing the bug, then please escalate. We will help you where we can.

Either way, our workaround solution seems to be working, so we will go with that.

Answer 1

Isaac Drew 0

Hi @Anonymous ,

We've changed to using Kudu's api/publish instead of api/zipdeploy (this is equivalent to "ZipDeploy/MSDeploy via Azure CLI or Azure DevOps." which you suggested).

This seems to restart the app correctly when deploy finishes, and our webjobs work after this.

0 comments

Share via

Continuous webjob cannot start after zipdeploy

1 answer

Your answer