Azure CosmosDb returning HTTP 503 Service Unavailable in North Europe region

Question

Azure CosmosDb returning HTTP 503 Service Unavailable in North Europe region

Benjamin Hubert 20

We're getting this response, for one, but not both of our CosmosDb instances, and we didn't change any configuration or do anything
On twitter it looks like we're not the only ones experiencing this.

Response status code does not indicate success: ServiceUnavailable (503); 
Substatus: 20003; 
ActivityId: a03231d5-d54c-456c-b2de-e89745fb8eda; 
Reason: (GatewayStoreClient Request Timeout. Start Time UTC:04/01/2025 08:47:27; Total Duration:71498.5995 Ms; Request Timeout 65000 Ms; Http Client Timeout:65000 Ms; Activity id: a03231d5-d54c-456c-b2de-e89745fb8eda;);;
Diagnostics:
{
    "Summary": {
        "GatewayCalls": {
            "(0, 0)": 3
        }
    },
    "name": "Typed FeedIterator ReadNextAsync",
    "start datetime": "2025-04-01T08:47:27.506Z",
    "duration in milliseconds": 71499.3936,
    "data": {
        "Client Configuration": {
            "Client Created Time Utc": "2025-03-28T08:40:32.2657144Z",
            "MachineId": "hashedMachineName:1df7ca7c-586d-0397-4dec-a478ae389542",
            "NumberOfClientsCreated": 1,
            "NumberOfActiveClients": 1,
            "ConnectionMode": "Direct",
            "User Agent": "cosmos-netstandard-sdk/3.41.0|1|X64|Debian GNU Linux 12 bookworm |.NET 9.0.3|N|",
            "ConnectionConfig": {
                "gw": "(cps:50, urto:6, p:False, httpf: False)",
                "rntbd": "(cto: 5, icto: -1, mrpc: 30, mcpe: 65535, erd: True, pr: ReuseUnicastPort)",
                "other": "(ed:False, be:False)"
            },
            "ConsistencyConfig": "(consistency: NotSet, prgns:[], apprgn: )",
            "ProcessorCount": 4
        },
        "Query Correlated ActivityId": "cf2836c4-5ef0-4167-b2a3-79a7b1b6aa6c"
    },
    "children": [
        {
            "name": "Create Query Pipeline",
            "duration in milliseconds": 71498.9804,
            "children": [
                {
                    "name": "Get Container Properties",
                    "duration in milliseconds": 0.0111,
                    "children": [
                        {
                            "name": "Get Collection Cache",
                            "duration in milliseconds": 0.001
                        }
                    ]
                },
                {
                    "name": "Gateway QueryPlan",
                    "duration in milliseconds": 0.2964
                },
                {
                    "name": "Microsoft.Azure.Cosmos.Handlers.RequestInvokerHandler",
                    "duration in milliseconds": 71498.8013,
                    "children": [
                        {
                            "name": "Microsoft.Azure.Cosmos.Handlers.DiagnosticsHandler",
                            "duration in milliseconds": 71498.7859,
                            "data": {
                                "System Info": {
                                    "systemHistory": [
                                        {
                                            "dateUtc": "2025-04-01T08:47:40.9263828Z",
                                            "cpu": 2.955,
                                            "memory": 2285804.000,
                                            "threadInfo": {
                                                "isThreadStarving": "False",
                                                "threadWaitIntervalInMs": 0.0299,
                                                "availableThreads": 32766,
                                                "minThreads": 4,
                                                "maxThreads": 32767
                                            },
                                            "numberOfOpenTcpConnection": 6
                                        },
                                        {
                                            "dateUtc": "2025-04-01T08:47:50.9266061Z",
                                            "cpu": 2.686,
                                            "memory": 2284416.000,
                                            "threadInfo": {
                                                "isThreadStarving": "False",
                                                "threadWaitIntervalInMs": 0.0473,
                                                "availableThreads": 32766,
                                                "minThreads": 4,
                                                "maxThreads": 32767
                                            },
                                            "numberOfOpenTcpConnection": 6
                                        },
                                        {
                                            "dateUtc": "2025-04-01T08:48:00.9268307Z",
                                            "cpu": 1.889,
                                            "memory": 2279540.000,
                                            "threadInfo": {
                                                "isThreadStarving": "False",
                                                "threadWaitIntervalInMs": 0.0373,
                                                "availableThreads": 32766,
                                                "minThreads": 4,
                                                "maxThreads": 32767
                                            },
                                            "numberOfOpenTcpConnection": 6
                                        },
                                        {
                                            "dateUtc": "2025-04-01T08:48:10.9271160Z",
                                            "cpu": 3.072,
                                            "memory": 2274568.000,
                                            "threadInfo": {
                                                "isThreadStarving": "False",
                                                "threadWaitIntervalInMs": 0.0383,
                                                "availableThreads": 32766,
                                                "minThreads": 4,
                                                "maxThreads": 32767
                                            },
                                            "numberOfOpenTcpConnection": 6
                                        },
                                        {
                                            "dateUtc": "2025-04-01T08:48:20.9274187Z",
                                            "cpu": 1.886,
                                            "memory": 2274736.000,
                                            "threadInfo": {
                                                "isThreadStarving": "False",
                                                "threadWaitIntervalInMs": 0.0491,
                                                "availableThreads": 32766,
                                                "minThreads": 4,
                                                "maxThreads": 32767
                                            },
                                            "numberOfOpenTcpConnection": 6
                                        },
                                        {
                                            "dateUtc": "2025-04-01T08:48:30.9276561Z",
                                            "cpu": 2.695,
                                            "memory": 2302260.000,
                                            "threadInfo": {
                                                "isThreadStarving": "False",
                                                "threadWaitIntervalInMs": 0.0492,
                                                "availableThreads": 32766,
                                                "minThreads": 4,
                                                "maxThreads": 32767
                                            },
                                            "numberOfOpenTcpConnection": 6
                                        }
                                    ]
                                }
                            },
                            "children": [
                                {
                                    "name": "Microsoft.Azure.Cosmos.Handlers.TelemetryHandler",
                                    "duration in milliseconds": 71498.7795,
                                    "children": [
                                        {
                                            "name": "Microsoft.Azure.Cosmos.Handlers.RetryHandler",
                                            "duration in milliseconds": 71498.7758,
                                            "children": [
                                                {
                                                    "name": "Microsoft.Azure.Cosmos.Handlers.RouterHandler",
                                                    "duration in milliseconds": 71498.7496,
                                                    "children": [
                                                        {
                                                            "name": "Microsoft.Azure.Cosmos.Handlers.TransportHandler",
                                                            "duration in milliseconds": 71498.7468,
                                                            "children": [
                                                                {
                                                                    "name": "Microsoft.Azure.Cosmos.GatewayStoreModel Transport Request",
                                                                    "duration in milliseconds": 71498.6906,
                                                                    "data": {
                                                                        "Client Side Request Stats": {
                                                                            "Id": "AggregatedClientSideRequestStatistics",
                                                                            "ContactedReplicas": [],
                                                                            "RegionsContacted": [],
                                                                            "FailedReplicas": [],
                                                                            "AddressResolutionStatistics": [],
                                                                            "StoreResponseStatistics": [],
                                                                            "HttpResponseStats": [
                                                                                {
                                                                                    "StartTimeUTC": "2025-04-01T08:47:27.5068648Z",
                                                                                    "DurationInMs": 497.441,
                                                                                    "RequestUri": "https://OMITTED.documents.azure.com/dbs/OMITTED/colls/OMITTED/docs",
                                                                                    "ResourceType": "Document",
                                                                                    "HttpMethod": "POST",
                                                                                    "ActivityId": "a03231d5-d54c-456c-b2de-e89745fb8eda",
                                                                                    "ExceptionType": "System.Threading.Tasks.TaskCanceledException",
                                                                                    "ExceptionMessage": "A task was canceled."
                                                                                },
                                                                                {
                                                                                    "StartTimeUTC": "2025-04-01T08:47:28.0043483Z",
                                                                                    "DurationInMs": 4999.5161,
                                                                                    "RequestUri": "https://OMITTED.documents.azure.com/dbs/OMITTED/colls/OMITTED/docs",
                                                                                    "ResourceType": "Document",
                                                                                    "HttpMethod": "POST",
                                                                                    "ActivityId": "a03231d5-d54c-456c-b2de-e89745fb8eda",
                                                                                    "ExceptionType": "System.Threading.Tasks.TaskCanceledException",
                                                                                    "ExceptionMessage": "A task was canceled."
                                                                                },
                                                                                {
                                                                                    "StartTimeUTC": "2025-04-01T08:47:34.0038520Z",
                                                                                    "DurationInMs": 65001.568,
                                                                                    "RequestUri": "https://OMITTED.documents.azure.com/dbs/OMITTED/colls/OMITTED/docs",
                                                                                    "ResourceType": "Document",
                                                                                    "HttpMethod": "POST",
                                                                                    "ActivityId": "a03231d5-d54c-456c-b2de-e89745fb8eda",
                                                                                    "ExceptionType": "System.Threading.Tasks.TaskCanceledException",
                                                                                    "ExceptionMessage": "A task was canceled."
                                                                                }
                                                                            ]
                                                                        }
                                                                    }
                                                                }
                                                            ]
                                                        }
                                                    ]
                                                }
                                            ]
                                        }
                                    ]
                                }
                            ]
                        }
                    ]
                }
            ]
        },
        {
            "name": "Query Response Serialization",
            "duration in milliseconds": 0.0138
        }
    ]
}

Christian Rasmussen 0 Reputation points

2025-04-01T09:51:26.9333333+00:00

Same issue but with our Azure storage account
Vijayalaxmi Kattimani 2,225 Reputation points Microsoft External Staff

2025-04-01T10:47:35.3066667+00:00

Hi Benjamin Hubert,

Greetings!

Please be informed that there was an outage in North Europe region today.

I request you to please refresh the portal and confirm if your database is back to online state or not, as the team is working on it.

Appreciate your patience while working with us on this request.

Thank you.
Benjamin Hubert 20 Reputation points

2025-04-01T10:48:58.97+00:00

I can confirm that the service it up and running again.

Accepted answer

0 additional answers

Your answer

Christian Rasmussen 0 Reputation points

2025-04-01T09:51:26.9333333+00:00

Same issue but with our Azure storage account
Vijayalaxmi Kattimani 2,225 Reputation points Microsoft External Staff

2025-04-01T10:47:35.3066667+00:00

Hi Benjamin Hubert,

Greetings!

Please be informed that there was an outage in North Europe region today.

I request you to please refresh the portal and confirm if your database is back to online state or not, as the team is working on it.

Appreciate your patience while working with us on this request.

Thank you.
Benjamin Hubert 20 Reputation points

2025-04-01T10:48:58.97+00:00

I can confirm that the service it up and running again.

Answer 1

Vijayalaxmi Kattimani 2,225 Microsoft External Staff

Hi Benjamin Hubert,

Thank you so much for your response.

I am glad to hear your issue has been resolved.

Since the Microsoft Q&A community has a policy that "The question author cannot accept their own answer. They can only accept answers by others", I'll repost your solution.

Please click "Accept" the answer as original posters help the community find answers faster by identifying the correct answer.

Issue: Azure CosmosDB returning HTTP 503 Service Unavailable in North Europe region.

Resolution: As mentioned earlier, your databases went into an unavailable state due to the North Europe region outage. Our internal team mitigated this issue, which is why your database is now back online.

I hope this has been helpful.

Your feedback is important so please take a moment to accept answers. If you still have questions, please let us know what is needed in the comments so the question can be answered. Thank you for helping to improve Microsoft Q&A! User's image

Please accept an answer if correct. Original posters help the community find answers faster by identifying the correct answer about the North Europe Region outage.

Vijayalaxmi Kattimani 2,225 Reputation points Microsoft External Staff

2025-04-02T09:15:59.14+00:00

Hi @Benjamin Hubert,

We would like to thank you for your feedback provided on survey. We found your recent engagement on this question was rated low. However, we were able to share some additional information that could helped in the later comment. Would you please re-evaluate your feedback.
Benjamin Hubert 20 Reputation points

2025-04-02T10:21:38.7366667+00:00

The only reason I am even on this board is because I wanted to create a support ticket on azure because CosmosDb went down. Since that feature has apparently been removed in favor of this public Q&A Forum that the Help section redirected me to, I just created my "ticket" here.

In reality, and I think you'll agree with this, there is absolutely nothing in this post that can be of use to anyone else than maybe some engineers at microsoft working with azure/cosmos.

This whole this is the result of an outage incident, with both the responsibility and means to fix it in the hands of azure engineers. There's nothing a user could do, nor any useful information they could gather from neither question nor answer.

The azure engineers fixed the thing, so now I am happy and my "ticket" has been resolved.

So I marked your answer as the "accepted answer" in order to close the thread, but when the UI then asked me to give feedback on whether this solved my problem, the answer was no, because it wasn't really a question to begin with, based on response times my coworker who wrote at the azure twitter account seemed to get actual support (i.e. they asked for subscription id to take a look at our resources), but before that even happened, it seems that they engineers at azure by them selves mitigated the outage incident.

So yes, this thread is closable. No, it didn't really help, as far as I can tell what helped was going on twitter with it.

I know that you're just an employee trying to help about, and this isn't a reflection of your work here, it's a reflection of the fact that I no longer have a way to create a real support ticket to report service outage/similar, and instead have to go to a "how can I do this and that"-type public forum.

I've been an azure user for almost 10 years, and the means I have to report issues / incidents have been limited more and more over time to the point where I wouldn't be surprised if at some point it's all on discord.

Anyway, sorry about the rant, and thanks for the polite answers.
Vijayalaxmi Kattimani 2,225 Reputation points Microsoft External Staff

2025-04-02T11:15:42.0333333+00:00

Hi @Benjamin Hubert,

We sincerely apologize for the inconvenience caused.

We would like to inform you that, Developer Support Plan does not include technical support since 06-Feb-2025. You must have a Standard support plan or higher (Standard, Professional Direct, or Premier) to open a technical support ticket.

We understand you have questions about the Developer support plan and how to engage with Microsoft Support Engineers to receive answers to your technical questions. Microsoft Q&A is our first-party question and answer platform where enthusiasts, MVPs, and other community experts can engage. The developer support plan for your subscription also ensures a new experience where you receive business hours access to Support Engineers on Microsoft Q&A with prioritized responses, as the first line of support.

Additionally, if you have issues pertaining to Azure billing or subscription management, you can always create a support request though the Azure Portal or by following this link.

We would be more than happy to assist you with any technical questions you have here on Q&A. Any time you post a new thread; you will receive a response within 8 business hours (see here for more details on the SLA and subscription benefits).

I hope this information helps. Please do let us know if you have any further queries.

Share via

Azure CosmosDb returning HTTP 503 Service Unavailable in North Europe region

0 additional answers

Your answer