New Flex Consumption function apps never start in our subscription — 7 dead hosts across 3 regions and 2 stacks, Premium control works (host runtime unreachable, no telemetry, no storage footprint)

Turbocharged Racing 0 Reputation points
2026-11-16T01:33:16.89+00:00

Summary

Every newly created Azure Functions Flex Consumption app in our subscription fails to start its host runtime — including minimal hello-world apps. Apps provisioned earlier in 2025 on the identical shape (same resource group, region, storage account) run normally, and an Elastic Premium (EP1) app created today as a control starts immediately. We are asking Microsoft to establish whether this is (i) a subscription-level provisioning restriction, (ii) a regional Flex Consumption infrastructure issue, or (iii) another platform-side failure — and what diagnostic surface exists when the host produces no telemetry and no storage footprint at all.

  • Subscription: c85e8e05-3c96-4a7d-8873-afb85195ba5c
  • Tenant: e57dbbb2-9548-4cac-a1ac-43b91dd6942b
  • Resource group: tcr-prod

The specimen matrix (all Flex Consumption FC1, all created 2026-11-13)

  1. ts-worker-func-staging — East US 2 — Python 3.11, managed identity — shared storage — dark (destroyed + recreated: identical)
  2. ts-worker-flex-probe — East US 2 — Python 3.11, key auth — shared storage — dark
  3. ts-worker-func-wus2 — West US 2 — Python 3.11, managed identity — fresh dedicated storage — dark
  4. probe cell A — West US 2 — Node 20 — dark
  5. probe cell B — Central US — Python 3.11 — dark
  6. ts-probe-ftp — East US 2 — Python 3.11, HTTP function — fresh storage — dark; host never started; both CLI deploy routes failed at the platform layer and the deployment container stayed empty

Control: ts-worker-func (production worker) — East US 2 — Python 3.11 — shared storage — runs normally (provisioned earlier in 2025)

Control: EP1 probe — East US 2 — Python 3.11, Elastic Premium — fresh storage — starts first try: root 200, functions enumerate, timer fires, telemetry flows

The signature (identical on every new Flex host)

  • ARM state "Running"; root URL returns HTTP 503 ("Function host is not running")
  • Functions enumeration returns "Encountered an error (InternalServerError) from host runtime"; sync triggers returns ServiceUnavailable
  • Zero Application Insights telemetry from the new hosts (connection strings verified against the working app's workspace)
  • Zero host storage footprint: no azure-webjobs-hosts container/instance, no host keys; restarts, cold stop/start, and destroy + recreate all reproduce it identically
  • Storage diagnostic logging on the shared account shows the new hosts never issue a single host-side storage operation — the only record is the deployment-side package fetch (GetBlob 200). The failure sits above the storage layer.
  • Azure Monitor metrics discriminate further: for two specimens, InstanceCount reads 1-2 and MemoryWorkingSet sits at ~459-469 MB for ~11-15 minutes, then stops — compute is assigned and a process starts, then dies in early host initialization, before any observable act. The healthy production host sustains 0.98-1.18 GB. The final read (why the process dies) sits in scale-controller logs customers cannot see.

Deployment correlation IDs (all status Success): staging twin instance 1 01e4dfcb-f115-4cac-b5ca-d4ccc1dea4df · twin instance 2 b47eb98c-1c24-4f5b-ab1d-6ea916429745 · probe 0edcfa65-6bb4-4ef3-a9a5-ce6fb18aa73c · test env graded deploy b91671a0-6fc4-4a41-9712-302aa2e2d0c0 (byte-identical package, sha256-verified).

What we have ruled out

  • Package/content: the production workload package imports and builds cleanly (remote build succeeds, deployment records Success); the hello-world package (one timer or one HTTP function, azure-functions only) fails identically. Not a stack fault (Python + Node both dark), not a regional fault (East US 2 / West US 2 / Central US all dark).
  • Storage auth form: managed identity and account-key forms both fail; the published azurerm empty-AccountKey poison string is absent.
  • RBAC/grants: identity principals hold Storage Blob Data Owner + Key Vault Secrets User, verified by-scope before and after recreate; DB principals registered with verified grant censuses.
  • App settings / Key Vault: 27/27 settings verified name- and value-level on the twin; every Key Vault reference resolves.
  • Service health: zero ServiceIssue events on the subscription in the window; the app's own detector headlines "Functions runtime is unreachable" over empty measurement tables. Resource Health is not available for Consumption-plan apps.

Question

What prevents a newly provisioned Flex Consumption host in this subscription from starting, when earlier-provisioned Flex apps run and a Premium app on the same shape starts immediately? Is there any customer-visible diagnostic surface for a host that dies in early initialization — no telemetry, no storage footprint, no host keys?

We are re-running a minimal provisioning attempt daily and will update this thread with each result (region/stack labeled) until a new Flex host starts.

Azure Functions
Azure Functions

An Azure service that provides an event-driven serverless compute platform.

0 comments No comments

2 answers

Sort by: Most helpful
  1. Turbocharged Racing 0 Reputation points
    2026-11-16T19:12:31.1533333+00:00

    Daily update — 2026-11-16: Region northeurope, stack Python 3.11 Flex Consumption FC1, app ts-flex-retry-6northeuropepy311. Verdict: HOST STARTED — root HTTP 200, functions enumeration answers. This is the first new Flex Consumption host to start in this subscription in this daily retry series. The app is being kept alive as evidence (not torn down). Previous daily retries and the original twin in eastus2 / other regions were dark with root 503 and host-runtime InternalServerError.

    Was this answer helpful?

    0 comments No comments

  2. AI answer

    2026-11-16T01:34:29.8066667+00:00

    The remaining actionable path is an Azure Functions support case requesting backend Flex Consumption startup and scale-controller investigation; the evidence provided does not establish which platform-side condition is terminating these hosts.

    The pattern is consistent with a host startup/platform allocation failure, a documented error class that includes placeholder specialization and container-allocation failures, but 503 Function host not running does not identify the underlying cause by itself.^1^

    Customer-visible diagnostics that remain

    1. Run the startup detector on one current specimen
      • Function App → Diagnose and solve problems
      • Search for Function App Down or Reporting Errors (Preview).
      • Export or capture every startup event, offline-history entry, hosting-plan result, and configuration result.
      This detector covers Flex, startup events, runtime validation, mandatory settings, SyncTrigger failures, host-name collisions, hosting setup, and scaling behavior.^1^
    2. Query workspace-level resource logs, not only Application Insights Flex documentation identifies FunctionAppLogs as the fallback when Application Insights is absent or incomplete, including host/control-plane failures. It also identifies AzureActivity for start, stop, list-keys, and SyncTrigger operations.^2^
         union isfuzzy=true FunctionAppLogs, AzureActivity
         | where TimeGenerated > ago(24h)
         | where _ResourceId has "/providers/Microsoft.Web/sites/<APP_NAME>"
            or ResourceId has "/providers/Microsoft.Web/sites/<APP_NAME>"
         | project TimeGenerated, Type, OperationName,
                   ActivityStatusValue, ResultType, Message, CorrelationId
         | order by TimeGenerated asc
      
      An empty result is itself useful escalation evidence: record the exact UTC interval during which Automatic Scaling Instance Count and Memory working set showed the short-lived process. Those metrics are documented Flex signals for instance allocation and process resource use.^3^
    3. Do not expect Kudu/SCM logs on Flex Microsoft’s Functions troubleshooting documentation explicitly excludes Flex Consumption from Kudu log availability.^1^ If the host exits before emitting Application Insights or resource logs, there is no documented customer-facing equivalent of the platform’s startup/scale-controller logs.
    4. Check Flex quota once as a discriminator
      • Function App → Diagnose and solve problems
      • Search for Flex Consumption Quota
      • Select each affected region and export current quota plus historical usage.
      Flex quota is subscription-and-region specific and does not include Premium capacity. Quota exhaustion can delay or fail Flex deployments while still allowing app resources to be created, so the successful EP1 control does not test this limit.^4^ Because the failures span three regions and instances were allocated, quota is a check—not an established diagnosis.

    Support request scope and payload

    Open Support + Troubleshooting → Create a support request from an affected Function App. Classify it under Azure Functions, app down/startup or provisioning, and request escalation to the Flex Consumption platform team.

    Include:

    • One surviving app resource ID per region, plan FC1, stack/version, instance memory, and creation timestamp in UTC.
    • A tight UTC reproduction window showing:
      • successful ARM deployment;
      • first 503;
      • SyncTrigger failure;
      • instance-count/memory appearance and disappearance.
    • Deployment operation IDs and Activity Log correlationId values. Obtain the latter from Resource group → Activity log → event → JSON → correlationId.^5^
    • Detector export, quota output, resource-log query results, metric screenshots/CSV, and deployment command with verbose output.
    • The cross-region/cross-stack matrix and the earlier Flex plus current EP1 controls.
    • Explicit request: correlate the UTC windows with Flex specialization, worker-startup, container-allocation, deployment-controller, and scale-controller logs, and check subscription-level Flex provisioning enforcement.

    Microsoft’s support preparation guidance specifically asks for the deployment method, verbose CLI/PowerShell output, and deployment correlation details.^6^

    A backend subscription restriction is possible but not proven. A previous case required Microsoft to remove an automated App Service subscription restriction; however, that case failed plan creation/update, whereas these resources provision successfully and fail later during host initialization.^7^ Ask support to check this condition rather than treating it as the established cause.

    The available public material does not identify a matching November 2026 multi-region incident or a customer-side remediation for this exact successful-provisioning/early-host-death signature. Further daily app creation is unlikely to expose the terminating event; retaining one specimen per region with precise timestamps will provide better backend correlation.


    References

    1. Troubleshoot Azure Functions app down or reporting errors
    2. Create and manage function apps in the Flex Consumption plan (programming-language-typescript)
    3. Azure Functions monitoring data reference
    4. Azure Functions Flex Consumption plan hosting | Microsoft Learn
    5. Get operation correlation ID from Activity Log for Microsoft Discovery
    6. Resolve issues when creating, deleting, or restoring Azure Functions
    7. Function App deployment stuck with BadGateway error on Microsoft.Web/serverfarms resource - Microsoft Q&A
    AI-generated content may be incorrect. Read our transparency notes for more information.

    Was this answer helpful?

Your answer

Answers can be marked as 'Accepted' by the question author and 'Recommended' by moderators, which helps users know the answer solved the author's problem.