ACS Call Recording - silent BYOS export failure with permanent media data loss (suspected cross-region/HADR finalization); escalation requested

Question

Hello Azure Microsoft support,

We have a production data-loss incident on our ACS call recording and are requesting escalation to the ACS product/engineering team. On 2026-06-12T16:03:33.4998814Z a single recording failed to export its media (MP4) to our Bring Your Own Storage container; the failure was only observable through the standard RecordingFileStatusUpdated event after the fact, the media existed solely in 24-hour short-term storage, and that window has now lapsed. The recording is unrecoverable to us and we have had to disclose the loss to the affected customer - a direct trust breach.

This is might be potentially related to our prior report, Microsoft Q&A 5806072 ("Significant delay in Azure Communication Services (ACS) call recording BYOS export for specific recording chunks", March 2026): https://learn.microsoft.com/en-gb/answers/questions/5806072/significant-delay-in-azure-communication-services
In that case chunks of a long call were finalized and exported to BYOS roughly 7 hours late but were eventually delivered with no data loss; the conclusion was a service-side delay during recording finalization/export, with no documented latency guarantee. The current incident is the same delayed-finalization behavior, except the BYOS export never completed - so the delay became actual data loss. Therefore, this time it is more serious as we can't find a way how to recover data given that 4 days has passed.

ACS Resource is in West Europe. BYOS container was properly configured via externalStorage (recordingStorageKind=azureBlobStorage, recordingDestinationContainerUrl) in production and working for all calls for ~1 year.

<PII removed>

What happened:

The acsmetadata.json was written correctly to BYOS roughly 7 hours before the event, at .../20260612/6c0844c2-.../BlNjMifQ-.../0-acsmetadata.json. The MP4 was never written to BYOS and there are no traces in the StorageBlobLogs in Log Monitor, only Blob calls related acsmetadata.json. The RecordingFileStatusUpdated event arrived ~7 hours after recording end (versus ~5 minutes on healthy calls) and reported contentLocation, metadataLocation, and deleteLocation on short-term storage (as-storage.asm.skype.com). The 24-hour download window expired around 2026-06-13T16:03Z, so the media is now gone.

UPD: It's worth to note that BYOS documentation states [1]:

Failure to export recording file can occur due to incorrect configuration settings or service outages. In both cases, the failure notification includes the download url to obtain recordings. Download recordings from the location in failure notification within 24 hours.

However, in this specific case, the notification received was indistinguishable from a successful one. It had the same eventType, same sessionEndReason, and same benign endReason in the recordingChunks object (CallEnded). Therefore, our internal processing logic didn't detect any failure and didn't trigger an alert or logged an error, that could have allowed us to react faster and retrieve the media from the temporary storage.

One bizarre observation: export finalization in a region on a different continent: The failed recording's documentId carries the region prefix "jhb" (0-jhb-d8-...), which we read as Johannesburg in South Africa North (southafricanorth) - a different continent from our ACS resource and storage, both West Europe ("weu"). For contrast, a healthy recording on the same resource the same day exported cleanly to BYOS ~5 minutes after it ended, with contentLocation on our own blob. And there was another recording export 2 minutes later at 2026-06-12T16:05:38, that also successfully was exported to BYOS in West Europe with the recordingDestinationContainerUrl that we specify in externalStorage when initiating a StartRecording call. Only the affected one behaved this way.

This appears to indicate a cross-region HADR (High Availability / Disaster Recovery) failover kicked in during finalization for the affected call: the small metadata file was exported to BYOS from the primary (weu) path, finalization then relocated to the failover region (jhb), and the BYOS export leg to our storage failed there, leaving the media in short-term storage only. I concluded this based on different continent in the URL and this documentation paragraph[2].

Please confirm whether this reading is correct.

We are asking you to:

1. Escalate this as a production data-loss incident given the customer trust impact that we must explain.
2. Attempt recovery of the media object for documentId 0-jhb-d8-941030ee5c52fb57cdf45560896e329c from backend storage if it physically persists beyond the 24-hour download-URL expiry. I assume there is no chance given what is stated in the official documentation, but perhaps you still have some backups?
3. Confirm the root cause - whether an HADR/cross-region failover affected finalization for this call, and why the BYOS media export failed in the failover region while the metadata export from the primary region succeeded.

Also, please confirm what mitigation reliably prevents recurrence, since the only signal available to us is the already-failed, 24-hour-bounded event.

Let us know if you need additional details and context. I can provide more IDs in the private message to avoid sharing them here. Thank you in advance.

[1] https://learn.microsoft.com/en-us/azure/communication-services/quickstarts/call-automation/call-recording/bring-your-own-storage?pivots=programming-language-java#notification-and-action-for-recording-export-failure
[2] https://learn.microsoft.com/en-us/azure/communication-services/concepts/voice-video-calling/call-recording#known-issues

Kind regards,
Alexander