App Services on Linux rebooting because exceeded "memory limits"/"swap usage limits"

Question

Hello,

I currently have a functional and production Windows service plan hosting 4 Blazor server side applications (2 production and 2 staging).

I want to switch these applications to a Linux app service plan.

So, I created a Linux Service Plan (P0v3, 1 instance) and deployed these same applications there (4 staging but only 2 used for tests). During our tests, we noticed that our applications were crashing regularly. Yet there are no more than 2 simultaneous users (unlike under Windows with over 100 users on P1v3, 2 instances).

In the logs, we can see the following messages:

• "Container for ****_msiProxy site **** is unhealthy, Stopping site."
• "Stopping site **** because it is not healthy."
• "Stopping site **** because it exceeded memory limits."
• "Stopping site **** because it exceeded swap usage limits."

I saw that Docker is used for hosting the applications, so I tried the variables: WEBSITE_MEMORY_LIMIT_MB at 4096 and CONTAINER_AVAILABILITY_CHECK_MODE in ReportOnly, but without success. The app service plan is around 98% memory usage... Is Linux so much more memory-consuming than windows ?

Should we consider a larger scaling for Linux in production?

Thank you in advance for your help.

Best regards,

Rémy B.

Answer 1

@Rémy Bertrand Thanks for reply!

To elaborate-

Low swap space (<=800MB) detected -

Swap space in Linux is an extension of the physical memory (RAM) on to the host disk. When RAM is full and system needs more memory resources, inactive pages are moved out of RAM into swap space.

Common Causes of Low swap space

• High number of Apps on the App Service Plan.
• Note that Production applications should run on a dedicated ASP to avoid resource contention with other applications.
• If Large Docker Images are used
• If you're using Web App for Containers, you'll want to choose the appropriate ASP to run your Docker Container.
• Application Code- To help with troubleshooting memory leaks, you can use an Application Performance Monitor (APM) to profile your application.

Also- Too many custom images (>=4) detected. This can lead to high host disk space usage. This impact site availability and the Disk space used to store custom images is limited. Once this disk space is consumed, container startup's will fail leading to availability issues.

Also its suggested to distribute your web app across multiple instances, Your webapp is currently configured to run on only one instance.    

Please Scale out and add additional instances. These instances are in different upgrade domains and hence will not be upgraded at the same time. While one worker instance is getting upgraded the other is still active to serve web requests.

With only one instance you can expect downtime because when the App Service platform is upgraded, the instance on which your web app is running will be upgraded. Therefore, your web app process will be restarted and will experience downtime.

Refer- Scale instance count manually or automatically

More Details-

Troubleshoot performance degradation is good place to configure application insights and monitoring your application.

If the issue reoccurs, we may need to dig deeper into the underlying root cause.

https://learn.microsoft.com/troubleshoot/azure/app-service/capture-memory-dumps-app-service#excessive-memory-consumption-scenario

Just sharing more info for some of the techniques and FAQs to handle resource -memory and storage related issues are covered in this doc scenario.

Additionally, The Ultimate Guide to Running Healthy Apps in the Cloud

 

Please let us know if further assistance required, happy to assist you.