2 App services with the same ip, but one can connect to vm, the other cannot

Ken Fly 41

In the same resource group, we have 2 app services. They are using the same docker image but start diff services, one is running rails server, the other is running sidekiq.

I checked the properties page of the two app services, they are exactly the same, the same virtual ip, the same Outbound IP and the same Additional Outbound IP.

However, only one app service, rails server, could connect to rabbitmq, which installed in the VM of the same resource group. The other app service, sidekiq, always be rejected when trying to connect the same rabbitmq.

Be rejected, I mean, when trying to telnet rabbitmq_ip 5672 is rejected

As I mentioned above, the two app services are using exactly the same ip config.

I am curious what could cause the diff?

ajkuma 22,086 Reputation points Microsoft Employee

2022-11-01T14:26:22.387+00:00

Just to clarify, has this worked fine before? Or are you facing this issue since the app was provisioned/made any changes prior to the issue happening?
What is the exact error message you receive when connecting to rabbitmq running sidekiq app?

Does this setup/connection works fine locally/outside of App Service?

Kindly see if setting the WEBSITES_PORT app setting to the port number helps. App Service forwards requests to that port in the container.

Add a startup script to Startup (doc) for the runtime stack.

I see a couple of similar discussion thread, when targeting services, not appending a port seems to work.

azure-client-connect-to-rabbitmq
How do I run Sidekiq on Azure?

Kindly let us know, I'll follow-up further.
Ken Fly 41 Reputation points

2022-11-02T05:05:46.513+00:00
@ajkuma Thanks very much for your reply.

Yes, it works before. But several months ago, they updated vm/rabbitmq ip. Since then, sidekiq could not connect to rabbitmq

As I mentioned, rails server, which has exactly the same ip config with sidekiq, has no issue to connect to rabbitmq. Also, my local pc, which has internal ip via vpn, could connect to rabbitmq without any issue.

Be connected, I mean telnet rabbitmq_ip 5672
Trying rabbitmq_ip...
Connected to rabbitmq_ip.
Escape character is '^]'.

From sidekiq, it is rejected like the below

telnet rabbitmq_ip 5672 Trying rabbitmq_ip... telnet: Unable to connect to remote host: Connection refused

I just tested, sidekiq has no issue to telnet google.com 80

Thanks for the links and suggestion provided.
I just tried without port solution. It does not work. Obviously, if we connect rabbitmq without port, we actually will use default rabbitmq port 5672.

sidekiq_alive and WEBSITES_PORT is an another interesting potential solution. Since it is prod server, I cannot make config change now.
I will try this weekend and update there then.

Thanks for all your suggestion
ajkuma 22,086 Reputation points Microsoft Employee

2022-11-03T08:20:16.507+00:00

KenFly-8462, Thanks for the follow-up and sharing additional info. Sure, please try those configs, and let us know how it goes, we would be more than happy to assist you further.
Ken Fly 41 Reputation points

2022-11-07T00:21:37.467+00:00
@ajkuma I've tried to add sidekiq_alive and update WEBSITES_PORT to 7433, which is sidekiq_alive's web server port. But all in vain, the sidekip keeps restarting.

I have to revert all changes. For some reason, the sidekiq container cannot run sidekiq, sidekiq_alive and sshd at the same time.

Anyway, we run 2 containers in one app service by docker-compose. Although, we cannot ssh to sidekiq container but we can ssh to another webserver container. From the web server container, we cannot telnet to rabbitmq

root@114e4e954a85:~# telnet 1.2.3.4(rabbitmq_ip) 5672 Trying 1.2.3.4... telnet: Unable to connect to remote host: Connection refused
ajkuma 22,086 Reputation points Microsoft Employee

2022-11-08T10:23:57.167+00:00

KenFly-8462, Apologies you're still experiencing this issue.

Thanks for the following and sharing additional info. For a deeper investigation, I have followed-up with you privately.
Ken Fly 41 Reputation points

2022-11-23T10:29:21.017+00:00

Ok, I think I found the reason.

Our internal rabbitmq has internal ip only.

Our app app service Outbound Traffic has VNet integration, the internal ip ranges covering the rabbitmq ip
However, our worker app service Outbound Traffic has No VNet integration, and the tip info shows

Virtual network integration is only supported on Basic and higher plans. Virtual network integration is not supported with multi-container (Docker Compose) apps.

Our worker is multi-container (Docker Compose) apps. How can we do that?
ajkuma 22,086 Reputation points Microsoft Employee

2022-11-25T15:14:06.88+00:00

Thanks for the follow-up. As outlined in this document "Preview limitations" -it's currently a limitation : "VNET integration is not supported for Docker Compose scenarios".
Ken Fly 41 Reputation points

2022-12-02T01:56:46.87+00:00

Thanks for your confirmation.

We have an internal service, previously it has a public ip. However, due to security concern, we change it to private ip.

Now due to this limitation, worker cannot access to private ip rabbitmq anymore.

Our app function is broken now.

:(
ajkuma 22,086 Reputation points Microsoft Employee

2022-12-02T19:04:50.15+00:00

KenFly-8462, Apologies for any inconvenience with this. I completely understand the scenario and requirement here.

However, due to the limitation (VNET integration is not supported for Docker Compose scenarios), it' doesn't work.
If your requirement fits, you may run it in an ASE ( which can host Docker containers (Windows and Linux) + and, provides Isolation and secure network access)
Please check this doc for more info : App Service Environment overview