In a middle of running a durable function, I get a message in logs "DrainMode mode enabled" and the durable function suddenly stops in the middle.

Question

In a middle of running a durable function, I get a message in logs "DrainMode mode enabled" and the durable function suddenly stops in the middle.

How can I prevent this, so that atleast already running functions get finished? I saw that in DOTNET, there are cancellation tokens but how can I prevent "DrainMode mode enabled" from stopping already running functions in javascript?

Answer 1

@Pepouš Hoblik Firstly, apologies for the late reply. Welcome to Microsoft Q&A Forum, Thank you for posting your query here!

I see that you are encountering "DrainMode mode enabled" message and the durable function suddenly stops in the middle. You want to avoid this.

For Azure functions running on consumption and elastic premium plan (serverless), the scale controller decides the scaling behavior. Drain mode comes into picture when the scale controller decides to scale down a worker (during less load). At this time, it sends a request to the Admin API to enter drain mode. Then all currently executing Functions on that instance receive a cancellation token which the customer can handle.

https://learn.microsoft.com/en-us/azure/azure-functions/functions-dotnet-class-library?tabs=v2%2Ccmd#cancellation-tokens

Then once all currently executing invocations have stopped then we will stop all of the listeners to ensure that the worker is shutting down doesn’t handle any new invocations. There’s no way to avoid the cancellation tokens.

In JavaScript (NodeJS), there’s no direct equivalent to .NET’s cancellation tokens. However, you can handle graceful shutdown in Node.js by listening for the ‘SIGTERM’ and ‘SIGINT’ events and cleaning up any resources or finishing any ongoing tasks before allowing your application to exit.

process.on('SIGTERM', () => {
  console.log('Received SIGTERM, finishing ongoing tasks...');
  // Here you can add your cleanup logic
  process.exit(0);
});

process.on('SIGINT', () => {
  console.log('Received SIGINT, finishing ongoing tasks...');
  // Here you can add your cleanup logic
  process.exit(0);
});

This won’t prevent the Azure Functions host from scaling down your function, but it will allow your function to finish any ongoing tasks before it is stopped.

AFAIK, The drain mode doesn't seem to have caused the functions to stop. I would suggest you to follow the below steps by looking into the Diagnose and solve problems blade and run the "Function app down or reporting errors" detector during the time the issue was seen. This detector will help you identify what caused the function host to go down.

You can also enable the Application insight logs for the FunctionApp and check if there are any exceptions. You can enabled the AppInsights as explained here.

Hope this helps.

**

Please do not forget to "Accept the answer” and “up-vote” wherever the information provided helps you, this can be beneficial to other community members.