Problem type: Availability Problem subtype: Function App down or reporting errors Severity: C — Minimal impact (internal automation, no external users)

Matt O'Brien 0 Reputation points
2026-05-13T15:03:10.08+00:00

Likely related to maintenance event 4QL8-81Z (End of Routine Planned Maintenance for App Service in East US, completed earlier today). The Function App eklay-quote-agent is in East US and has been returning persistent 503 since deployment attempts that coincided with this maintenance window. The portal notification described "a brief drop in connectivity for a few seconds," but the host has been wedged at 503 for many hours and does not recover across restarts, redeploys, or a full delete-and-recreate of the Function App.

SYMPTOM

  • Site root https://eklay-quote-agent.azurewebsites.net returns HTTP 503 ServiceUnavailable continuously
  • az functionapp function list returns "Bad Request" (host management API unavailable)
  • Three timer-triggered functions in function_app.py are never invoked
  • No application logs surface in az webapp log tail or via Kudu

RESOURCE

  • Subscription: Pay-As-You-Go
  • Resource group: [PII removed]
  • Function App: [PII removed]
  • Region: East US
  • Plan: Linux Consumption, Python 3.11, v2 programming model

VERIFIED CONFIGURATION (all correct)

  • WEBSITE_RUN_FROM_PACKAGE = 1
  • FUNCTIONS_WORKER_RUNTIME = python
  • FUNCTIONS_EXTENSION_VERSION = ~4
  • AzureWebJobsStorage set and reachable (storage account [PII removed], all four required containers present: azure-webjobs-hosts, azure-webjobs-secrets, quote-agent-state, scm-releases)
  • AzureWebJobsFeatureFlags = EnableWorkerIndexing (required for Python v2 model)
  • linuxFxVersion = Python|3.11
  • Deployed package present in scm-releases: scm-latest-eklay-quote-agent.zip, ~28.78 MB, last modified 2026-05-13T01:56:23Z

STEPS ALREADY ATTEMPTED

  1. Deploy via az functionapp deployment source config-zip --build-remote true — host stayed 503
  2. Full recreate: deleted the Function App, created fresh, repopulated all settings — same 503
  3. Deploy via func azure functionapp publish eklay-quote-agent --python --build remote — Oryx remote build completed successfully, package uploaded to scm-releases, host still 503
  4. Set WEBSITE_RUN_FROM_PACKAGE=1 (was cleared by func publish) and restarted — still 503 after extended cold-start window
  5. SCM Basic Auth enabled; storage account reachability verified; all settings populated; linuxFxVersion confirmed

REQUEST

Please investigate the underlying host startup failure and provide the worker initialization error. The Kudu API and standard log channels are not surfacing diagnostic data on this plan, which is itself a clue that the platform is not in a healthy state. Specifically asking whether maintenance event 4QL8-81Z is implicated, and whether a platform-side recovery action is available.

Azure Functions
Azure Functions

An Azure service that provides an event-driven serverless compute platform.


1 answer

Sort by: Most helpful
  1. Matt O'Brien 0 Reputation points
    2026-05-21T17:18:48.74+00:00

    Hi Pravallika — thanks for the suggestions, closing the loop on this one.

    The issue was resolved on 2026-05-13. Quick post-mortem in case it's useful for others hitting a similar wedge:

    Diagnostics during the outage. The detectors you pointed to weren't reachable while the host was in this state — "Function App Down or Reporting Errors" in the Diagnose-and-Solve blade returned no data, Kudu file logging (host.json fileLoggingMode: Always) produced no startup logs because the worker never came up far enough to write any, App Insights traces was empty for the affected window, and az webapp log download returned an empty archive. az functionapp function list continued to return "Bad Request" against the live host. These channels all going silent simultaneously was itself the strongest signal that the host node was wedged at a layer below where customer-side instrumentation could see.

    Resolution. We didn't manage to recover the Linux Consumption instance. After confirming with a second redeploy that the host was not going to self-heal, we recreated the Function App on the Flex Consumption plan (FC1, East US) — same Python 3.11 code, same storage account, same identity wiring, fresh app resource. The new app picked up the package on first deploy, the three timer triggers registered immediately, and it's been running on its hourly/SOD/EOD schedule cleanly since.

    Takeaway. For future tickets in this shape: if a Linux Consumption Python app lands in a persistent 503 where Kudu, App Insights, and ARM management APIs are all dark at the same time, in our case the recovery path was to abandon the plan rather than repair the wedged instance. Flex Consumption is the modern Python target anyway, so the migration was overdue.

    Appreciate the prompt response on this. Marking as resolved.

    — Matt

    Was this answer helpful?

    0 comments No comments

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.