How to add alerts regards to Azure Application gateway backend health

Question

Hi Team, I have a doubt on adding alerts to the Azure application gateway for backend health. Whenever the backend is unhealthy, the mail to be triggered to the team mentioning the application is unhealthy. I have tried creating alerts but the mail gets triggered only once, I need it for triggering atleast once a day till the issue is resolved. Can anyone provide me a solution for the same.

Answer 1

Hope doing good!

To set up alerts for Azure Application Gateway backend health that trigger multiple times until the issue is resolved, you can use Azure Monitor Alerts with a combination of Log Analytics and an action group. Here are the steps to achieve this:

1. Create a Log Analytics Workspace:
   • If you don't already have one, create a Log Analytics Workspace in the Azure portal and if you have one you can use existing workspace

Configure Diagnostic Settings:

• Configure Diagnostic Settings for your Application Gateway to send health probe status data to the Log Analytics Workspace.
• You can do this by going to your Application Gateway settings, selecting "Diagnostics settings," and then configuring the appropriate diagnostics settings to send data to the Log Analytics Workspace.

Create a Log Analytics Query Alert:

• In the Log Analytics Workspace, create a query that checks the health probe status. For example, you can use a query like this to detect unhealthy backend instances:

sqlCopy code
Heartbeat
| where ResourceType == "APPLICATIONGATEWAYS" and Resource == "YourApplicationGatewayName"
| where ResourceId contains "/backendhealth/"
| where Status_s != "Healthy"

• Save the query.

Create an Action Group:

• In the Azure portal, go to "Monitor" -> "Alerts" -> "Manage actions groups."
• Create a new action group, and configure it to send email notifications to your team members.
• Add this action group to your Application Gateway alerts.

Create an Alert Rule:

• Go to "Monitor" -> "Alerts" -> "New alert rule."
• Configure the alert rule with the following settings:
• Resource: Your Application Gateway
• Condition: Use the Log Analytics query you created in step 3.
• Action group: Select the action group you created in step 4.
• Evaluation frequency: Set it to your desired frequency (e.g., every 5 minutes).
• Severity: Set the appropriate severity.
• Enable the alert rule.

Set Alert Suppression (Optional):

• If you don't want to receive alerts continuously for the same issue within a short timeframe, you can set up alert suppression. This prevents multiple alerts for the same issue within a specified period.

With these steps, Azure Monitor Alerts will continuously evaluate the query you created in Log Analytics and trigger email notifications to your team whenever the backend health condition matches the criteria defined in your query. The alerts will continue to trigger until the issue is resolved or the alert suppression period expires.

MS doc link for ref --> https://learn.microsoft.com/en-us/azure/azure-monitor/logs/quick-create-workspace?tabs=azure-portal

https://learn.microsoft.com/en-us/azure/azure-monitor/logs/monitor-workspace

Answer 2

@Murali R

Welcome to the Microsoft Q&A Platform. Thank you for reaching out & I hope you are doing well.

I understand that you would like to configure alerts for an Application Gateway unhealthy backend Pools.

Before this, I would like to highlight certain important concepts with App Gateway and how App gateway considers a backend healthy or not.

• If a single FQDN BackendTarget in a single BackendPool resolves to “n” IPs, Then the total number of hosts in metrics becomes “n”
  • i.e., Unhealthy hosts + Healthy hosts = n
• This “n” also includes the IPV6 addresses, even though App gateway will not probe this IPV6 address (at the moment)
• Furthermore, BackendHealth section considers a backendTarget healthy only if all the “n” IPs are reachable and working.
  • So, if there were a IPV6 or a non-working IP in that “n” addresses, the BackendHealth from Portal would simply show as Unhealthy only

Because of the above reason, we must only consider the Unhealthy Host Count or Healthy Host Count for configuring any such Alerts

I would suggest you to go with Healthy Host Count.

The idea is to

• Make a note of the Healthy Host count during a normal moment
• Use this value as the Alert threshold.
• Should the value become less than this value, at any point in time, we must trigger an Alert.
• And to make sure the Alert keeps on triggering every time and not just only once, do not enable Automatically resolve alerts
  • 

Now, you can configure the Alert to work with both Log Queries or Azure Monitor Metrics Tab.

I shall explain the Metrics tab here.

Metrics Tab :

• Display Healthly Host count and make sure you filter using the backend HTTP settings
• 
• Make sure you set the LocalTime set as Last 30 minutes.
• Click on "New Alert Rule"
• Make sure you set the Threshold value to your default Healthy Count
• As you can see from the above diagram, my default is 1 and it went to 0 once my backend stopped working
• 
• Set the frequency as you want
  • 
• Under Actions Tab, Create a Action Group
  • create Mail or SMS as Notification
    • Feel free to skip Action in the Action Group
• And finally under "Details" Tab, under "Advanced"
  • 

This should trigger an Notification via SMS/Email once in every 15 minutes and will the use the average for last 30 minutes (As per my configuration).

Kindly let us know if this helps or you need further assistance on this issue.

NOTE:

• You must come up with the actual Healthy Host Count for a normal App gateway depending upon your configuration.
• Based on this, you must set the threshold for the Alert.

Thanks,

Kapil

---

Please don’t forget to close the thread by clicking "Accept the answer" wherever the information provided helps you, as this can be beneficial to other community members.