Azure Function requests to Cosmos periodically getting blocked

Question

Azure Function requests to Cosmos periodically getting blocked

James 1

We're on an Azure function consumption plan that connects with a CosmosDB. We have a vnet setup so that CosmosDB doesn't need to be publicly accessible. however, we're still getting periodic IP blocking with the following error message:

"One or more errors occurred. (Request originated from client IP ##.###.##.## through public internet. This is blocked by your Cosmos DB account firewall settings"

Help me understand why this happens and how to resolve it.

Anonymous

2021-04-06T01:50:06.503+00:00

@James

Please go through this document for details regarding this. Please let us know if you have any further questions regarding this.

Regards
Navtej S
James 1 Reputation point

2021-04-06T13:22:59.607+00:00

Thanks for your response. However, it seems like an issue in Azure itself because we have the vnet setup for communication between the azure function and CosmosDB. Both services are on the same vnet/subnet in their network/firewall settings so why would we be getting intermittent blocking at a public IP level when it's an internal network request? I could be missing an important detail but currently this behavior doesn't make sense...

3 answers

Your answer

Anonymous

2021-04-06T01:50:06.503+00:00

@James

Please go through this document for details regarding this. Please let us know if you have any further questions regarding this.

Regards
Navtej S
James 1 Reputation point

2021-04-06T13:22:59.607+00:00

Thanks for your response. However, it seems like an issue in Azure itself because we have the vnet setup for communication between the azure function and CosmosDB. Both services are on the same vnet/subnet in their network/firewall settings so why would we be getting intermittent blocking at a public IP level when it's an internal network request? I could be missing an important detail but currently this behavior doesn't make sense...

Answer 1

Anonymous

@James

We checked with our team internally and they have proposed the following

"The problem looks like on the Azure Functions end. We are not sure how you have set up VNET integration but they noticed that you are using Consumption plan for Az Functions, The consumption plan doesn’t support VNET integration.

You could review these to check you have set things up correctly:

Configure virtual network based access for an Azure Cosmos account | Microsoft Learn
Azure Functions networking options | Microsoft Learn

Please let us know if you need any further info.

Regards
Navtej S

James 1 Reputation point

2021-04-08T18:16:34.963+00:00

Thanks for this... We are currently on the premium consumption plan which technically should support vnet like any other app service. My team still believes there is a intermittent issue (maybe as the consumption plan scales in/out) that affects this vnet therefore causing blocks when it attempts to connect with Cosmos.

Any insights there that would be helpful for us?
Anonymous

2021-04-09T22:12:46.42+00:00

@James

We will definitely check with team for the insights but at this point as it requires to look into your telemetry, Please raise an issue on support as well. Please do share the ticket number so that we can get traction for the issue.

Regards
Navtej S
James 1 Reputation point

2021-04-12T14:54:03.337+00:00

Thanks for the help. Last week we did end up adding our full list of public IPs to the CosmosDB firewall whitelist which seems to have resolved the intermittent issue. BUT according to our understanding we shouldn't have to do this if we're using vnets. We have a band-aid in place that is helping and appreciate the microsoft team digging further to see if this is related to auto scaling in the consumption plan or some other reason why the vnet isn't fully covering our cross region DB connections to Cosmos when the request is coming from varying IPs on an azure function consumption plan.

Answer 2

@James

We are glad the issue is resolved for now. We did get a further response from the team on the same lines where a review of the architecture is needed and they have shared few documents as well:

If they are on Premium then yes VNET integration is possible, but I still think this is really an Az Functions issue or maybe the way they have set up VNET integration, rather than a Cosmos issue. Cosmos wouldn’t block those requests if they were sourced from within the VNET so I think they should review their network architecture to ensure the function app is calling Cosmos DB service endpoint in the same VNET or via private link if appropriate.

See:
Azure Functions networking options | Microsoft Learn
Integrate app with Azure Virtual Network - Azure App Service | Microsoft Learn
Configure virtual network based access for an Azure Cosmos account | Microsoft Learn

Hope these documents help and provide more info.

Answer 3

devopsfj 261

Did you ever get a resolution to this? I have a similar issue with AKS.

James 10 Reputation points

2024-01-19T14:52:43.7466667+00:00

We ended up switching to a premium azure function.

Share via

Azure Function requests to Cosmos periodically getting blocked

3 answers

Your answer