edit: Alexander confirmed that "Switching off automatic scale up option has resolved the issue."
To better assist you on this, what App Service Plan (ASP) are you leveraging?
What Azure subscription type are you using? In which region is your webapp provisioned?
I understand from your issue description - the issue persists even after scaling up the app plan and integrating Application Insights, which hasn't revealed any suspicious activity during the periods in question. The memory consumption and processor utilization are also within normal ranges.
if you haven't done, to isolate the issue:
Review for any auto-scaling rules or health check configurations. Checkout this doc: Best practices for autoscale
If your application relies on other services or databases, ensure that they are not experiencing issues or performing maintenance tasks at the same times.
To isolate the issue, you may leverage App Service diagnostics from Azure Portal> Navigate to your App Service app in the Azure Portal.
In the left navigation, click on Diagnose and solve problems
- Review - “Configuration and Management and “Availability and Performance”
Enable and review logs to fetch more details about the error: #enable-application-logging-linuxcontainer and Access log files