Why does my scheduled Cloud Function fail from time to time?

Question

I have an Azure Cloud Function to reset my game's Playfab Leaderboard. The function is triggered by Playfab automation at 2am UTC every day. The function has been running for 4 months. It normally executes successfully, but every 10-12 days it will fail. When that happens, the leaderboard is not updated and my users play the previous day's game, which is a bad experience.

This is a log stream from a successful execution:

2023-10-31T02:00:58Z [Verbose]   Initiating background SyncTriggers operation 
2023-10-31T02:01:04Z   [Information]   Executing StatusCodeResult, setting HTTP status code 204 
2023-10-31T02:01:08Z   [Verbose]   Request successfully matched the route with name 'ResetDailyHighestScoreLeaderboard2' and template 'api/ResetDailyHighestScoreLeaderboard2' 
2023-10-31T02:01:08Z   [Information]   Executing 'ResetDailyHighestScoreLeaderboard2' (Reason='This function was programmatically called via the host APIs.', Id=7ecc9ee4-ce2b-4b69-b983-bc4f7c067fee) 
2023-10-31T02:01:08Z   [Information]   >> Start ResetDailyHighestScoreLeaderboard 
2023-10-31T02:01:09Z   [Information]   >> Reset Leaderboard: Daily Highest Score Classic 2, v125, DeactivationTime: (null) 
2023-10-31T02:01:09Z   [Information]   >> Reset Leaderboard: Daily Highest Score Expert 2, v125, DeactivationTime: (null) 
2023-10-31T02:01:09Z   [Information]   Executed 'ResetDailyHighestScoreLeaderboard2' (Succeeded, Id=7ecc9ee4-ce2b-4b69-b983-bc4f7c067fee, Duration=734ms)

and this is a log of a failed run:

 Connected!

2023-11-06T02:59:58Z   [Verbose]   Initiating background SyncTriggers operation
2023-11-06T03:00:07Z   [Information]   Executing StatusCodeResult, setting HTTP status code 204
2023-11-06T02:59:58Z   [Verbose]   Initiating background SyncTriggers operation 
2023-11-06T03:28:13Z   [Verbose]   Initiating background SyncTriggers operation 
2023-11-06T03:29:21Z   [Information]   Host Status: {   "id": "resetdailyhighestscoreleaderboar",   "state": "Running",   "version": "4.27.5.21549",   "versionDetails": "4.27.5+096e48198ef4cc3259d9ad0f45891a74992825a6",   "platformVersion": "100.0.7.517",   "instanceId": "4dd8e98f154af9bddba1c3c11f33cd46040b2baa6b9b058b1ce9a8580076f5a2",   "computerName": "10-30-12-0",   "processUptime": 452338,   "functionAppContentEditingState": "Unknown" } 
2023-11-06T03:29:21Z   [Information]   Host Status: {   "id": "resetdailyhighestscoreleaderboar",   "state": "Running",   "version": "4.27.5.21549",   "versionDetails": "4.27.5+096e48198ef4cc3259d9ad0f45891a74992825a6",   "platformVersion": "100.0.7.517",   "instanceId": "4dd8e98f154af9bddba1c3c11f33cd46040b2baa6b9b058b1ce9a8580076f5a2",   "computerName": "10-30-12-0",   "processUptime": 452426,   "functionAppContentEditingState": "Unknown" }
 2023-11-06T03:28:13Z   [Verbose]   Initiating background SyncTriggers operation 
2023-11-06T03:29:21Z   [Information]   Host Status: {   "id": "resetdailyhighestscoreleaderboar",   "state": "Running",   "version": "4.27.5.21549",   "versionDetails": "4.27.5+096e48198ef4cc3259d9ad0f45891a74992825a6",   "platformVersion": "100.0.7.517",   "instanceId": "4dd8e98f154af9bddba1c3c11f33cd46040b2baa6b9b058b1ce9a8580076f5a2",   "computerName": "10-30-12-0",   "processUptime": 452338,   "functionAppContentEditingState": "Unknown" } 
2023-11-06T03:29:21Z   [Information]   Host Status: {   "id": "resetdailyhighestscoreleaderboar",   "state": "Running",   "version": "4.27.5.21549",   "versionDetails": "4.27.5+096e48198ef4cc3259d9ad0f45891a74992825a6",   "platformVersion": "100.0.7.517",   "instanceId": "4dd8e98f154af9bddba1c3c11f33cd46040b2baa6b9b058b1ce9a8580076f5a2",   "computerName": "10-30-12-0",   "processUptime": 452426,   "functionAppContentEditingState": "Unknown" }

It looks like the function did not start executing. The Playfab request timed-out and failed. Result, no leadership reset. Again this only happens somewhat randomly but about once every 10-12 days. The function itself is very simple, it just calls adminApi.IncrementPlayerStatisticVersionAsync(newRequest) from the PLayFab API.

Any ideas of what could be going wrong? Is it a matter of resource allocation?

Answer 1

After working with an Azure support engineer we seem to have found a solution. I was using the free Azure tier, which apparently can cause your function to start with a delay from time to time. After switching my plan to "Function Premium" the cloud script seems to be running correctly.

In your Azure account, navigate to 'Function app>App Service plan>Change App Service plan>Choose 'Function Premium' as your new Plan type'.

Unfortunately you will incur a cost, oh well.