Azure Container Instances Randomly Pulling and Restarting unchanged Image

Question

I received notification this morning at 08:02 that there was an availability error in my application. The App Service could not reach the container via the ip address I had configured, 10.24.2.5. I login to the portal to check and it seems the ip address of my container instances has changed to 10.24.2.4. According to the documentation

If the container group is stopped, started, or restarted, the container group's private IP is subject to change. If this happens, you will need to update the application gateway configuration.

I do not know why my container group was stopped, started or restarted, however I noticed a couple of events in the log which coincide with the time the incident occurred.

This is strange to me, as there was nothing I did overnight to prompt azure container instances to try and fetch a new image. The image it's pulling is hosted in an Azure Container Registry, and from the portal we can see that this image has not changed since June 2020 and so pulling should not cause any stop/start/restart to occur.

Can someone please explain why in the middle of the night, azure container instances takes it upon itself to try and pull a new version of my image, restarts itself and then gets a new ip address?

Answer 1

We've had the same issue but our restart policy was set to always, we are changing it to onfailure and also specifying the IP for the container instance to avoid changing IPs.

https://learn.microsoft.com/en-us/azure/container-instances/container-instances-troubleshooting

Answer 2

I'm facing the same issues. We have several containers, all deployed with RestartPolicy="OnFailure", but they keep randomly pulling the image and therefore containers are restarted and most of the time changing their IP addresses. We have others processes that depend on that private IP address so if it changes, this breaks things.

The image hasn't changed so I don't understand what else could be forcing to pull the image again.

Answer 3

Anybody get to the bottom of why this happened ?

Answer 4

The same happened to our ACI setup as well, just two days ago, for unknown reasons. One container group got restarted but this activity is not shown in the activity log. We store the logs outside in a log analytics workspace so I could see the applications restart sequence as well as the uptime command showed that the container got restarted a few hours ago. I wonder what triggers this restart, so far I have not received a satisfying answer from Microsoft

Answer 5

Same over here (unknown cause of container restart yesterday). The event log doesn't square with the container state summary in the attached screenshot (that also shows the earlier random image pulling last month). None of the container alerts were tripped either (includes start/stop/restart/start events).