Allow networking between scaled-out instances

Question

Allow networking between scaled-out instances

Botond Beres 0

We're experiencing a networking issue with containers running in Azure. Our application consists of multiple containers that need to communicate with each other directly. However, we've discovered that these containers are being assigned IP addresses in different link-local networks:
- Container A: 169.254.254.2:8080
- Container B: 169.254.130.3:8080

These IP addresses are in different subnets within the APIPA/link-local range, making direct communication impossible without additional routing.

Business Impact:
This is blocking deployment of our product which needs to be platform-agnostic. We cannot rely on Azure-specific networking services to solve this as our solution must work identically across multiple cloud and on-premises environments.

Technical Details:
- The containers need direct network communication without intermediaries
- We are NOT using AKS or other orchestration-specific solutions
- Our product must maintain identical networking behavior across different environments (AWS, GCP, on-premises, etc.)

Expected Behavior:
Containers should either:
1. Be assigned IP addresses in the same subnet, allowing direct communication, OR
2. Have proper routing configured automatically to allow cross-subnet communication within the 169.254.x.x range

Questions:
1. Why are containers being assigned IP addresses in different subnets within the link-local range?
2. Is there a configuration option to ensure all containers receive IP addresses in the same subnet?
3. How can we resolve this without relying on Azure-specific services or network configurations?

We need a solution that works solely at the container/network level without relying on Azure-specific infrastructure, as this application must run identically in multiple environments.

Thank you for your assistance.

Botond Beres 0 Reputation points

2025-04-02T13:28:33.5366667+00:00

P.S.: We tried using the environment variables HOSTNAME:PORT provided in the container which provided us with something like "app-name-6fd8287f:8080" but that lead to a DNS resolution error basically telling me there is no DNS configured.
Alekhya Vaddepally 1,670 Reputation points Microsoft External Staff Moderator

2025-04-02T14:51:13.7133333+00:00

Hi Botond Beres,
When the containers are not clearly connected to a Vnet, Azure provides them a link-local IP address. Without Vnet, there is no built-in network infrastructure to facilitate communication between containers, resulting in each container working in isolation with its own link-local address.

To enable direct communication between your containers, it is necessary to deploy them within the same Vnet. This approach provides an integrated networking environment, ensuring that all containers obtain IP addresses within the same subnet and can originally communicate

https://techcommunity.microsoft.com/blog/startupsatmicrosoftblog/from-chaos-to-clarity-simplifying-your-networking-with-azure-container-apps/4034625?
if you have any further concerns or queries, please feel free to reach out to us.
Botond Beres 0 Reputation points

2025-04-02T15:57:18.6733333+00:00

Hi Alekhya,

thanks for your quick reply, I believe I we are using a vnet, please have a look at the screenshot below. We're simply setting scale-out to manual with instance count 2.
This is the default configuration created by Azure when we set this app up:

We haven't changed anything but I did find the environment variable "WEBSITE_PRIVATE_IP" just now and tried it, this now results in the following error: Exception during request: Connection refused (10.0.1.252:8080). So I believe we are in the right network but the "PORT" environment variable seems to be the port to be used with HOSTNAME not WEBSITE_PRIVATE_IP for internal communication?

Please advise
Botond Beres 0 Reputation points

2025-04-02T16:00:22.51+00:00
Alekhya Vaddepally 1,670 Reputation points Microsoft External Staff Moderator

2025-04-03T04:29:06.1033333+00:00

Hi Botond Beres,

Since your app is posted in VNET, each container example gets a private IP. Variable webstry_private_ip represents this assigned IP. However, internal communication between examples does not occur automatically on these IP until clearly configured.

The error connection refused (10.0.1.252:8080) states that the application is trying to join a specific example using its personal IP but cannot hear on that port or allow direct communication in that way.

In an Azure App service setup, internal container-to-concenter communication is usually using hostname instead of the website_Pet_IP. This means that your application should use the hostname of the container instead of manually referring to the IPS.

Use Hostname for internal communication instead of websty_private_ip. Azure App services uses DNS-based service search for the route requests between service examples. Connect using the recommended approach:

PHP-template

http: // <hostname>: <port>

Where the <hostname> is an automatically assigned name of the example.

Make sure the application hears at the correct port. Port environment variable is the one that should be used while forcing the service inside the container. The application should be heard at 0.0.0.0: <port> rather than binding a specific private IP.

Check the networking settings. If necessary, verify that there is no network security groups (NSG) rules or restrictions that block traffic between examples within the VNET.

https://learn.microsoft.com/en-us/azure/app-service/networking-features
Botond Beres 0 Reputation points

2025-04-03T16:41:20.8433333+00:00

Hi Alekhya,
as mentioned before: "P.S.: We tried using the environment variables HOSTNAME:PORT provided in the container which provided us with something like "app-name-6fd8287f:8080" but that lead to a DNS resolution error basically telling me there is no DNS configured."

This is the exact error when trying to use hostnames: Name or service not known (app-name-76d71fe4:8080)

Also, here is the rest of the subnet settings:
As far as I can see there is nothing set that should block communication between instances.
Botond Beres 0 Reputation points

2025-04-03T21:49:12.1933333+00:00

As for what the container is listening on, in Azure Kudu I can see this Environment Variable: ASPNETCORE_URLS = 0.0.0.0:8181 which now causes even more confusion because when I try to reach the other instance internally, nothing replies on 8181, in fact, netstat comes back with: tcp6 0 0 :::8080 :::* LISTEN 1097/dotnet So its listening on IPV6 and Port 8080 instead of IPv4 and 8181 and is ignoring the env var? There is nothing in the project that would tell it to bind to port 8080, no launchsettings.json, no appsettings.json, nothing, so this must be Azure's doing.

Joke of the Month: Turns out I cant post replies or answers if they contain "h t t p", because some genius at Microsoft thought that's a smart limitation for an ONLINE Q&A platform.
So assume "h t t p : / /" before the above ASPNETCORE_URLS value.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
KarishmaTiwari-MSFT 20,777 Reputation points Microsoft Employee Moderator

2025-04-04T21:33:29.23+00:00

@Botond Beres We apologize for the issues that you have encountered with the Q&A platform. We understand how frustrating this can be and are actively working with our platform team to identify the cause and find a solution. We want to assure you that we're committed to doing everything we can to improve your experience going forward. Your feedback is incredibly important to us.

Your developer support plan includes business hours access to Support Engineers on Microsoft Q&A, ensuring you receive prioritized responses as the first line of support. We can also connect with you via private message to gather more details as needed and assist further.

Our technical experts are currently investigating your technical issue based on the information you have shared and will share an update soon. Thanks for your patience and understanding while we are working towards resolving this.

Priority community support (PCS) for Azure developer support plan customers FAQs
Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-04T23:03:08.6266667+00:00

Hi @Botond Beres first off, we are deeply sorry to hear that your experience on this platform has not been the best. We've shared your valuable feedback with the Microsoft Q&A engineering team.

Secondly, thank you for providing the additional details about your current setup/requirements. Based on your scenario requirements, it's clear that achieving platform-agnostic direct communication between your containers is a primary goal. Is that correct?

While using the WEBSITE_PRIVATE_IP and port 8080 might enable communication within the current Azure App Service environment, it's important to understand that this approach is specific to Azure and will not meet your requirement for the solution to work identically across multiple cloud and on-premises environments.

To achieve platform-agnostic direct communication between your containers, especially when considering the need for consistent behavior across diverse environments, we advise considering a container orchestration platform like Azure Kubernetes Service (AKS).

Leveraging AKS can address your requirements for platform-agnostic communication at the orchestration layer, direct communication, and avoiding Azure-specific networking. While AKS runs within an Azure VNet, the internal networking for service discovery and communication between Pods is managed by Kubernetes itself, not directly by Azure-specific networking services like Azure Load Balancer (though Azure may provide the underlying network infrastructure).

We hope this helps. Please let us know if you have further questions.

-Grace
Harshitha Veeramalla 1,306 Reputation points Microsoft External Staff Moderator

2025-04-05T04:07:23.2033333+00:00

Hi @Botond Beres, what is the Azure resource you are using Azure App Service or Azure Container App?
Arko 4,150 Reputation points Microsoft External Staff Moderator

2025-04-05T11:20:47.7033333+00:00
Hello Botond Beres,

Based on your description so far, I understand that you're facing networking issues where containers running in Azure (via App Services or Container Apps) are assigned different link-local (APIPA) IPs like 169.254.x.x — making direct communication between them impossible.

Your primary goal is to enable platform-agnostic direct container-to-container communication, without depending on Azure-specific networking features like WEBSITE_PRIVATE_IP or App Service DNS, so that it behaves identically across Azure, AWS, GCP, and on-premises correct?

To achieve this goal, you can use Kubernetes (can be aks, eks, gke or even open source k8s so platform agnostic) as suggested by Grmacjon-MSFT

Now since this is Microsoft QnA support, I can show you using Azure Kubernetes

I am using python and flask code to deploy my container for this example. You can use your own preferred choice of code.

Since you mentioned container a and b in your question, I am creating app-a/app.py and app-b/app.py

app-a/app.py

from flask import Flask import requests app = Flask(__name__) @app.route('/') def call_app_b(): response = requests.get("http://app-b:8080") return f"App-A received: {response.text}" app.run(host="0.0.0.0", port=8080)

app-b/app.py

from flask import Flask app = Flask(__name__) @app.route('/') def hello(): return "Hello from App-B" app.run(host="0.0.0.0", port=8080)

Here's my Dockerfile (same for both)

FROM python:3.9-slim WORKDIR /app COPY requirements.txt . RUN pip install -r requirements.txt COPY . . EXPOSE 8080 CMD ["python", "app.py"]

and my requirement file

flask requests

Now you need to create a Kubernetes cluster. As I said, this can be anywhere on Azure, AWS, GCP, Redhat, Opensource, etc

and you will also need a container registry to hold your containerized image. Again, this could be azure container registry (acr) or AWS's Elastic container registry (ecr) or Google container registry (gcr) or Redhat's quay registry.

I am showing using azure container registry and azure Kubernetes. Choose the number of nodes and resources based on your application's requirement.

az acr create --resource-group arkorg --name arkacr12345 --sku Basic --admin-enabled true az aks create --resource-group arkorg --name ark-aks-cluster --node-count 1 --generate-ssh-keys az aks get-credentials --resource-group arkorg --name ark-aks-cluster az aks update --name ark-aks-cluster --resource-group arkorg --attach-acr arkacr12345

Build and push images

docker build -t arkacr12345.azurecr.io/app-a:latest ./app-a docker build -t arkacr12345.azurecr.io/app-b:latest ./app-b docker push arkacr12345.azurecr.io/app-a:latest docker push arkacr12345.azurecr.io/app-b:latest

Deploy to your Kubernetes cluster. I will be using these YAML files for my example-

app-a.yaml

apiVersion: apps/v1 kind: Deployment metadata: name: app-a spec: replicas: 1 selector: matchLabels: app: app-a template: metadata: labels: app: app-a spec: containers: - name: app-a image: arkacr12345.azurecr.io/app-a:latest ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: app-a spec: selector: app: app-a ports: - port: 8080

app-b.yaml

apiVersion: apps/v1 kind: Deployment metadata: name: app-b spec: replicas: 1 selector: matchLabels: app: app-b template: metadata: labels: app: app-b spec: containers: - name: app-b image: arkacr12345.azurecr.io/app-b:latest ports: - containerPort: 8080 --- apiVersion: v1 kind: Service metadata: name: app-b spec: selector: app: app-b ports: - port: 8080

and apply the same-

kubectl apply -f app-b.yaml kubectl apply -f app-a.yaml

Now port-forward and test

kubectl port-forward deployment/app-a 18080:8080 curl http://localhost:18080

Therefore, in this example, AKS (or any Kubernetes platform) assigns Pods IPs within the same routable overlay subnet.

Internal DNS is automatically handled via CoreDNS.

No reliance on WEBSITE_PRIVATE_IP, Azure DNS, or link-local addressing. Fully reproducible across AWS, GCP, Azure, on-prem, or even Docker Desktop with Kubernetes enabled.

Hope this answers your query.
Arko 4,150 Reputation points Microsoft External Staff Moderator

2025-04-07T03:48:09.6266667+00:00

Following up on this Question Botond Beres. Hope my suggestion was useful for you.
NGANDU-BISEBA Gabriel 35 Reputation points

2025-04-09T11:32:54.1533333+00:00

Unfortunately, unless I misunderstand your requirements, what you are asking for is not possible on App Service.

The environment variable "WEBSITE_PRIVATE_IP" that has been floating around is not something that can work because it is an environment variable populated by the VNET integration. Vnet integration is a feature for outbound communication. Meaning that this private IP will only accept incoming traffic if it is the one who initiated a connection.

What you are looking for are private endpoints. Which give a routable IP inside a VNET. However, the private endpoints are shared between each instance of your app. So assuming that you have 100 PEs linked to a single app and 2 instances. Each of two instances can use any of the 100 private IPs at any time. There is no way to pin 1 private endpoint for 1 scaled out instance.

From what I understand from your requirements, it is critical that each instance must have a unique private IP, which is impossible.
Botond Beres 0 Reputation points

2025-04-09T19:26:26.5166667+00:00

@NGANDU-BISEBA Gabriel "each instance must have a unique private IP, which is impossible." I would be very surprised if that was the case, obviously the IPs within a subnet have to be unique otherwise you have a IP address conflict and no traffic can be routed properly. In said subnet each instance should get their own IP address just like in a physical network and according to the Azure website that is the case, see here:

Clearly 2 out of 251 IP addresses are in use, but ifconfig on either instance does not show those as configured for some reason. I understand that the IP address cannot be "fixed"/static (which might be what you meant with "unique") but DNS should solve that problem. In our case though DNS doesn't seem to be working properly either (when using HOSTNAME:PORT). Please see my latest comment below for more details.

2 answers

Your answer

Botond Beres 0 Reputation points

2025-04-02T13:28:33.5366667+00:00

P.S.: We tried using the environment variables HOSTNAME:PORT provided in the container which provided us with something like "app-name-6fd8287f:8080" but that lead to a DNS resolution error basically telling me there is no DNS configured.
Alekhya Vaddepally 1,670 Reputation points Microsoft External Staff Moderator

2025-04-02T14:51:13.7133333+00:00

Hi Botond Beres,
When the containers are not clearly connected to a Vnet, Azure provides them a link-local IP address. Without Vnet, there is no built-in network infrastructure to facilitate communication between containers, resulting in each container working in isolation with its own link-local address.

To enable direct communication between your containers, it is necessary to deploy them within the same Vnet. This approach provides an integrated networking environment, ensuring that all containers obtain IP addresses within the same subnet and can originally communicate

https://techcommunity.microsoft.com/blog/startupsatmicrosoftblog/from-chaos-to-clarity-simplifying-your-networking-with-azure-container-apps/4034625?
if you have any further concerns or queries, please feel free to reach out to us.
Botond Beres 0 Reputation points

2025-04-02T15:57:18.6733333+00:00

Hi Alekhya,

thanks for your quick reply, I believe I we are using a vnet, please have a look at the screenshot below. We're simply setting scale-out to manual with instance count 2.
This is the default configuration created by Azure when we set this app up:

We haven't changed anything but I did find the environment variable "WEBSITE_PRIVATE_IP" just now and tried it, this now results in the following error: Exception during request: Connection refused (10.0.1.252:8080). So I believe we are in the right network but the "PORT" environment variable seems to be the port to be used with HOSTNAME not WEBSITE_PRIVATE_IP for internal communication?

Please advise
Botond Beres 0 Reputation points

2025-04-02T16:00:22.51+00:00
Alekhya Vaddepally 1,670 Reputation points Microsoft External Staff Moderator

2025-04-03T04:29:06.1033333+00:00

Hi Botond Beres,

Since your app is posted in VNET, each container example gets a private IP. Variable webstry_private_ip represents this assigned IP. However, internal communication between examples does not occur automatically on these IP until clearly configured.

The error connection refused (10.0.1.252:8080) states that the application is trying to join a specific example using its personal IP but cannot hear on that port or allow direct communication in that way.

In an Azure App service setup, internal container-to-concenter communication is usually using hostname instead of the website_Pet_IP. This means that your application should use the hostname of the container instead of manually referring to the IPS.

Use Hostname for internal communication instead of websty_private_ip. Azure App services uses DNS-based service search for the route requests between service examples. Connect using the recommended approach:

PHP-template

http: // <hostname>: <port>

Where the <hostname> is an automatically assigned name of the example.

Make sure the application hears at the correct port. Port environment variable is the one that should be used while forcing the service inside the container. The application should be heard at 0.0.0.0: <port> rather than binding a specific private IP.

Check the networking settings. If necessary, verify that there is no network security groups (NSG) rules or restrictions that block traffic between examples within the VNET.

https://learn.microsoft.com/en-us/azure/app-service/networking-features
Botond Beres 0 Reputation points

2025-04-03T16:41:20.8433333+00:00

Hi Alekhya,
as mentioned before: "P.S.: We tried using the environment variables HOSTNAME:PORT provided in the container which provided us with something like "app-name-6fd8287f:8080" but that lead to a DNS resolution error basically telling me there is no DNS configured."

This is the exact error when trying to use hostnames: Name or service not known (app-name-76d71fe4:8080)

Also, here is the rest of the subnet settings:
As far as I can see there is nothing set that should block communication between instances.
Botond Beres 0 Reputation points

2025-04-03T21:49:12.1933333+00:00

As for what the container is listening on, in Azure Kudu I can see this Environment Variable: ASPNETCORE_URLS = 0.0.0.0:8181 which now causes even more confusion because when I try to reach the other instance internally, nothing replies on 8181, in fact, netstat comes back with: tcp6 0 0 :::8080 :::* LISTEN 1097/dotnet So its listening on IPV6 and Port 8080 instead of IPv4 and 8181 and is ignoring the env var? There is nothing in the project that would tell it to bind to port 8080, no launchsettings.json, no appsettings.json, nothing, so this must be Azure's doing.

Joke of the Month: Turns out I cant post replies or answers if they contain "h t t p", because some genius at Microsoft thought that's a smart limitation for an ONLINE Q&A platform.
So assume "h t t p : / /" before the above ASPNETCORE_URLS value.
Deleted

This comment has been deleted due to a violation of our Code of Conduct. The comment was manually reported or identified through automated detection before action was taken. Please refer to our Code of Conduct for more information.
KarishmaTiwari-MSFT 20,777 Reputation points Microsoft Employee Moderator

2025-04-04T21:33:29.23+00:00

@Botond Beres We apologize for the issues that you have encountered with the Q&A platform. We understand how frustrating this can be and are actively working with our platform team to identify the cause and find a solution. We want to assure you that we're committed to doing everything we can to improve your experience going forward. Your feedback is incredibly important to us.

Your developer support plan includes business hours access to Support Engineers on Microsoft Q&A, ensuring you receive prioritized responses as the first line of support. We can also connect with you via private message to gather more details as needed and assist further.

Our technical experts are currently investigating your technical issue based on the information you have shared and will share an update soon. Thanks for your patience and understanding while we are working towards resolving this.

Priority community support (PCS) for Azure developer support plan customers FAQs
Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-04T23:03:08.6266667+00:00

Hi @Botond Beres first off, we are deeply sorry to hear that your experience on this platform has not been the best. We've shared your valuable feedback with the Microsoft Q&A engineering team.

Secondly, thank you for providing the additional details about your current setup/requirements. Based on your scenario requirements, it's clear that achieving platform-agnostic direct communication between your containers is a primary goal. Is that correct?

While using the WEBSITE_PRIVATE_IP and port 8080 might enable communication within the current Azure App Service environment, it's important to understand that this approach is specific to Azure and will not meet your requirement for the solution to work identically across multiple cloud and on-premises environments.

To achieve platform-agnostic direct communication between your containers, especially when considering the need for consistent behavior across diverse environments, we advise considering a container orchestration platform like Azure Kubernetes Service (AKS).

Leveraging AKS can address your requirements for platform-agnostic communication at the orchestration layer, direct communication, and avoiding Azure-specific networking. While AKS runs within an Azure VNet, the internal networking for service discovery and communication between Pods is managed by Kubernetes itself, not directly by Azure-specific networking services like Azure Load Balancer (though Azure may provide the underlying network infrastructure).

We hope this helps. Please let us know if you have further questions.

-Grace
Harshitha Veeramalla 1,306 Reputation points Microsoft External Staff Moderator

2025-04-05T04:07:23.2033333+00:00

Hi @Botond Beres, what is the Azure resource you are using Azure App Service or Azure Container App?
Arko 4,150 Reputation points Microsoft External Staff Moderator

2025-04-07T03:48:09.6266667+00:00

Following up on this Question Botond Beres. Hope my suggestion was useful for you.
NGANDU-BISEBA Gabriel 35 Reputation points

2025-04-09T11:32:54.1533333+00:00

Unfortunately, unless I misunderstand your requirements, what you are asking for is not possible on App Service.

The environment variable "WEBSITE_PRIVATE_IP" that has been floating around is not something that can work because it is an environment variable populated by the VNET integration. Vnet integration is a feature for outbound communication. Meaning that this private IP will only accept incoming traffic if it is the one who initiated a connection.

What you are looking for are private endpoints. Which give a routable IP inside a VNET. However, the private endpoints are shared between each instance of your app. So assuming that you have 100 PEs linked to a single app and 2 instances. Each of two instances can use any of the 100 private IPs at any time. There is no way to pin 1 private endpoint for 1 scaled out instance.

From what I understand from your requirements, it is critical that each instance must have a unique private IP, which is impossible.
Botond Beres 0 Reputation points

2025-04-09T19:26:26.5166667+00:00

@NGANDU-BISEBA Gabriel "each instance must have a unique private IP, which is impossible." I would be very surprised if that was the case, obviously the IPs within a subnet have to be unique otherwise you have a IP address conflict and no traffic can be routed properly. In said subnet each instance should get their own IP address just like in a physical network and according to the Azure website that is the case, see here:

Clearly 2 out of 251 IP addresses are in use, but ifconfig on either instance does not show those as configured for some reason. I understand that the IP address cannot be "fixed"/static (which might be what you meant with "unique") but DNS should solve that problem. In our case though DNS doesn't seem to be working properly either (when using HOSTNAME:PORT). Please see my latest comment below for more details.

Answer 1

Botond Beres 0

Thank you all for your responses. I appreciate your efforts, but I need to clarify a few important points:

I'm specifically using Azure App Service with scale-out capability, not Azure Container Apps.
I do not wish to migrate to Kubernetes or AKS. Suggesting a completely different service architecture is not a solution to my current issue. If Azure App Service officially supports scale-out functionality (which it clearly does, as per the UI), then there should be a way to enable direct communication between those scaled instances.
To reiterate my original question: Why are container instances within the same App Service and VNet being assigned IP addresses in different subnets that cannot communicate with each other? This seems like a fundamental networking configuration issue.
I've already confirmed:

We have a VNet configured
Environment variables like HOSTNAME don't resolve (DNS error: "Name or service not known")
WEBSITE_PRIVATE_IP produces "Connection refused" errors
The container is listening on IPv6 port 8080 (via netstat), despite ASPNETCORE_URLS being set to 0.0.0.0:8181

To me it looks like that the environment variables HOSTNAME:PORT provided in the container which provided us with something like "app-name-6fd8287f:8080" (which in turn should translate into 10.0.1.xxx IP) would work is it wasn't for the DNS resolution error: "Name or service not known".

What I need is specific guidance on how to properly configure the VNet or App Service networking settings to allow direct communication between instances. This is a basic networking requirement for any scaled application and should be possible without requiring architectural changes.

Could someone from your team please address the specific networking configuration required, rather than suggesting we switch to entirely different Azure services?

Thank you.

Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-07T19:07:30.1033333+00:00

@Botond Beres Thanks for the clarification. We are checking internally with the engineering team to get more insights on your question and will get back to you when we hear back from them.

Best,

Grace
Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-07T20:05:37.3033333+00:00

@Botond Beres is this a linux web app? also can you please confirm the app framework you're running? Did you follow this step to configure it to allow connections? https://learn.microsoft.com/en-us/dotnet/orleans/deployment/deploy-to-azure-app-service#configure-private-port-count-using-azure-cli

Typically, the .NET cross node connections are called Orleans app. Here is an Azure doc explaining how to setup Orleans: https://learn.microsoft.com/en-us/dotnet/orleans/deployment/deploy-to-azure-app-service#configure-host-networking
Botond Beres 0 Reputation points

2025-04-07T20:38:35.1633333+00:00
@Grmacjon-MSFT We're deploying using:

Publishing model: Code

Runtime Stack: Dotnetcore - 8.0

As for the Orelans setup, Orleans seems to be more of an architectural design of the app rather than an actual Azure resource we would add, right?
As mentioned before, our app already handles leader election internally and our app is not deploying a client and server separately but instead any instance can be either Leader or Follower once said leader election is done.
The only thing I can see in the article you provided is this which we didn't try is this:

IPAddress.Parse(builder.Configuration["WEBSITE_PRIVATE_IP"]!);

I'm going to give that a go but what I'm missing for this approach is the WEBSITE_PRIVATE_PORTS variable as described in the article.
I can't see an environment variable called WEBSITE_PRIVATE_PORTS, I only have WEBSITE_PRIVATE_IP. Why?

As for your second link, I don't see any specific network configuration to allow communication between instances in that article besides the "usual" setup of a Vnet which Azure did for us (screenshots of those settings are available above).
Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-07T22:16:02.9566667+00:00
@Botond Beres I should have specified, did you configure this setting to at least 2?

This is what should populate the website_private_ports variable.

az webapp config set -g '<resource-group-name>' --subscription '<subscription-id>' -n '<app-service-app-name>' --generic-configurations '{"vnetPrivatePortsCount": "2"}'

If you have already added that configuration, then we will need to troubleshoot this issue offline with you.

Thanks,

Grace
Botond Beres 0 Reputation points

2025-04-09T01:38:54.12+00:00

Thank you for your quick reply. Where do I specifically set "vnetPrivatePortsCount" in the UI? For a "Logic-App (Standard)" I found that it should be under configuration like so: https://i.imgur.com/0K3AqM2.png but for my Web App i don't see some of those options incl. "VNet Private Ports" or "Web sockets".
Botond Beres 0 Reputation points

2025-04-09T19:05:12.0266667+00:00
Thank you @Grmacjon-MSFT for the guidance. I've executed the command you provided. The command completed successfully, and I can see that "vnetPrivatePortsCount" is now set to 2 in the JSON output. However, after restarting the WebApp, the WEBSITE_PRIVATE_PORTS environment variable is still not appearing in either instance.

Looking at both instances via Kudu and running ifconfig, neither shows an IP address in the 10.0.1.0 subnet, despite the subnet itself showing 2 IPs as being used:

Instance1:

Instance2:

This suggests there's a fundamental disconnect between the VNet configuration and the actual network interfaces assigned to the containers.

At this point, given that:

We've followed the recommended configuration steps

We've been troubleshooting this issue for over a week

This is blocking our deployment

I would like to request escalation of this issue to the level of the previous Developer Subscription support (phone support). We need this resolved urgently as it's holding up our production deployment.

To summarize the core issue: We're running a .NET 8 application in Azure App Service with scale-out enabled, and we need direct communication between instances. Despite configuring the VNet and now setting vnetPrivatePortsCount as instructed, the instances are not properly connecting to the VNet subnet for inter-instance communication.

Thank you for your continued assistance.
Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-09T19:07:09.5666667+00:00
Hi @Botond Beres some properties for web apps are not exposed from the Azure Portal UI so you will need to use your preferred deployment method, REST/ARM/CLI/etc

here is an example from CLI that I shared earlier :

az webapp config set -g '<resource-group-name>' --subscription '<subscription-id>' -n '<app-service-app-name>' --generic-configurations '{\"vnetPrivatePortsCount\": "2"}'

See the schema definition here: https://learn.microsoft.com/en-us/azure/templates/microsoft.web/sites/config-web?pivots=deployment-language-arm-template

let me know if the above works for you
Botond Beres 0 Reputation points

2025-04-09T19:10:06.5833333+00:00

You must've just missed my update, I tried the command, please see above.
Grmacjon-MSFT 19,301 Reputation points Moderator

2025-04-10T01:54:31.0633333+00:00

@Botond Beres thanks for the update. I just sent you a private message so we can troubleshoot this issue offline.

Answer 2

@Botond Beres

Please read my comment again. What you are looking at is vnet integration, a feature that only works for outbound communication.

The IPs bound to vnet integration can only receive traffic from TCP connections they initiated. This means that it will not help you to make instances of the same app on the same app service plan talk to each others. See the vnet integration as a NAT gateway inside your vnet if you will.

To have an inbound private IP inside a vnet that any other resource can connect to privately you must use private endpoints. However, private endpoints are shared between all instances of the same app. Meaning that you can not pin any private endpoints to a specific instance of the web app.

If you have 2 private endpoints and 2 scaled out instances of the same app. Private endpoint 1 may transfer traffic to any of the two instances and private endpoint 2 will do the same. Which is why I say that your individual instances having a unique private IP is impossible. For outbound, you have a NAT gateway. For inbound, you have a pool of IP address(es) through private endpoints that cannot be pinned.

Now obviously you know you requirements better than I do. But from what I understand, what you should be looking for to solve your issues are private endpoints and not vnet integration.

Share via

Allow networking between scaled-out instances

2 answers

Your answer