Health Probe in Azure Application Gateway for Azure Storage Account with Network ACLs and Service End Points based access, and IAM/RBAC through Azure AD

Question

We have a scenario where we need to put Azure API Management (deployed in internal VNET mode) and a few Azure Storage Accounts behind an Azure Application Gateway with WAFv2.

The Storage Accounts have Network ACLs that allow access via the subnet of the Azure Application Gateway only, and corresponding Service Endpoint configured on that Subnet. The IAM / RBAC on those storage account is through Azure AD based App Registrations and Managed Identities. We are able to set it up and access the Storage Account for uploading and downloading blobs, so from a functional standpoint its working as expected.

However, we are not able to find a proper Health Probe configuration for Azure Storage Accounts that works in this set up and returns a HTTP 2xx response code. For now, to make it work, we have configured the Azure Application Gateway Health Probes to access <storageaccountname>.blob.core.windows.net/ via the HTTPS protocol and treat HTTP 400 or 409 as healthy response codes. However, that's more like a hack to make it work.

The health probe settings for it are shown below for quick reference:

The reason we have to allow both 409 and 400 as expected response codes for the health probe is due to the fact that up till 30th Jan 2023, we used to get 409 status code from our storage account blob service URL. But all of a sudden on 30th Jan 2023 morning, our health probes started reported unhealthy status for storage accounts and stopped the traffic, as the storage accounts started returning 400 status code instead of 409. So, we have to implement another stop gap measure and treat both 400 and 409 as healthy status code to make things functionally work. However, this raises the question that what's the enduring solution for Azure Application Gateway health probes for Azure Storage Accounts, which have network ACLs in place, service end point based access, and IAM/RBAC set up using Azure AD App Registrations/Managed Identities.

Please note that the set up of Azure API Management (deployed in internal VNET mode) behind Azure Application Gateway is well documented and available on Microsoft website; and the health probe for that on a special path of /status-0123456789abcdef i.e., "<api management gateway URL>/status-0123456789abcdef" over HTTPS is working perfectly fine and returns a 200 status code as indication of healthy status.

So, wanted to know if there is any specific health probe URL for Azure Storage Account which is accessible even when the Storage Account has access from specific subnets and service end points and has IAM/RBAC through Azure AD (can be through App Registrations or Managed Identities) and returns a 2xx status code as an indication of healthy status?

Answer 1

@ZQadir

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

I understand that you would like to know the best methods to configure health probes for a Storage Account.

If you directly access the FQDN of a storage account, you should get a 4xx Status Code. This is expected.

The proper way to configure this is by specifying the absolute path of the resource, for e.g.

https://<storageaccount>.blob.core.windows.net/<container>/<healthcheckfile.html>

I see you are using AAD Authentication. This complicates the health probe check.

Please note currently Azure Application gateway cannot use managed identities to access other services, at least not directly.

We can overcome this by using SAS Tokens/URL

You can create a new container with a healthCheck file, specifically for HealthCheck and either

• Use SAS
• or allow private subnet access (and not use AAD)

Use SAS URL:

• Here, you will be required to create a SAS URL
• And create a custom Probe and the backend path should be , "/container/healthcheckfile.html/?<SAS>"
• The HealthChecks should now succeed.

Normal private subnet access

• Here, we do not impose any authentication, just allow Private subnet access.

I hope was able to help.

Thanks,

Kapil

---

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.