Database Failed

Question

Database Failed

Mark Davis 0

We have observed repeated failures on our production database starting from 11th September 2025. The database has failed every day for the past five consecutive days. This pattern is concerning for service continuity and may indicate an underlying resource or configuration issue.

Impact:

Service reliability is at risk due to repeated outages

Potential disruption to dependent applications and users

Risk of data access delays or downtime during business-critical operations

Request: Please investigate the Resource Health for this database instance to identify the root cause of these repeated failures. We would like to understand:

The specific cause(s) of the failures

Any related alerts or incidents in the service backend

Recommended remediation steps to prevent recurrence

Additional Details:

Service: common-rm-qa-psql-01

Region: North Europe

Failure Dates: 11th September 2025 – 15th September 2025 (daily)

09/15/2025	1 health event(s)		1 health event(s)
09/15/2025	1 health event(s)1 health event(s)
09/14/2025	2 health event(s)2 health event(s)
09/13/2025	1 health event(s)1 health event(s)
09/12/2025	4 health event(s)4 health event(s)
09/11/2025	11 health event(s)11 health event(s)

Environment: QA

Priority: High – QA Impact

Anonymous

2025-09-15T11:41:42.57+00:00
Hi Mark Davis,

Here are some steps you can take to investigate the issue:

Review Alerts and Incidents: Navigate to the Alerts section in the Azure portal. Check if there were any alerts related to your database service during the failure dates. This might give insights into what went wrong.

Database Logs: Access the database logs for any errors or abnormal activities leading up to and during the failure periods. You can find these logs in the Azure portal under the SQL databases resource.

Configuration Check: Verify the database configurations, including scaling settings and connection strings, to ensure everything is correctly configured.

Investigate Resource Limits: Ensure that your database isn't hitting resource limits (like DTUs or storage) that could cause it to fail. You can check resource metrics in the Azure portal.

Can you please confirm, the requested details below.

Can you provide specific error messages or logs you might have received during the database failures?

What type of database (e.g., SQL server, PostgreSQL, etc.) are you using, and what service tier or plan is currently assigned?

Reference document:

https://learn.microsoft.com/en-us/azure/azure-sql/database/resource-health-to-troubleshoot-connectivity?view=azuresql

Could you also please share the requested details via private message so we can proceed with further investigation?

Thanks!

Kalyani

1 answer

Your answer

Anonymous

2025-09-15T11:41:42.57+00:00

Hi Mark Davis,

Here are some steps you can take to investigate the issue:

Review Alerts and Incidents: Navigate to the Alerts section in the Azure portal. Check if there were any alerts related to your database service during the failure dates. This might give insights into what went wrong.

Database Logs: Access the database logs for any errors or abnormal activities leading up to and during the failure periods. You can find these logs in the Azure portal under the SQL databases resource.

Configuration Check: Verify the database configurations, including scaling settings and connection strings, to ensure everything is correctly configured.

Investigate Resource Limits: Ensure that your database isn't hitting resource limits (like DTUs or storage) that could cause it to fail. You can check resource metrics in the Azure portal.

Can you please confirm, the requested details below.

Can you provide specific error messages or logs you might have received during the database failures?

What type of database (e.g., SQL server, PostgreSQL, etc.) are you using, and what service tier or plan is currently assigned?

Reference document:

https://learn.microsoft.com/en-us/azure/azure-sql/database/resource-health-to-troubleshoot-connectivity?view=azuresql

Could you also please share the requested details via private message so we can proceed with further investigation?

Thanks!

Kalyani

Answer 1

Hi Mark Davis,

As part of our discussion over the private messaging, our internal team identified the server reported as unhealthy.

Mitigation: The PostgreSQL container which was repeatedly restarting due to input/output errors related to the data drive. Despite these container failures, the data drive itself remained in a normal state. Various related services and containers were restarted as part of the recovery process, and the overall server status was eventually restored to a healthy state.

The database is now back online, so you should not see any further failures as the backend team has mitigated the issue. As you confirmed, the database is accessible again. Since you have requested the RCA for the unavailability, I am actively following up with the PG team and will share the update with you as soon as I receive it.

Thanks for your patience!

Regards,

Kalyani

Share via

Database Failed

1 answer

Your answer