Did you check the logs from both the Azure SQL Database and the App Service for patterns and specific times of failures?
You need to verify that the managed identity token is refreshed before it expires and that all instances of the App Service are correctly configured to use the managed identity, especially if scaling or instance restarts are involved.
Also, check if your managed identity has the necessary permissions on the SQL Database.
Verify if you have followed the steps mentioned in the standard documentation without any issue:
and successfully provisioned a Microsoft Entra administrator https://learn.microsoft.com/en-us/azure/azure-sql/database/authentication-aad-configure?view=azuresql&tabs=azure-powershell#provision-azure-ad-admin-sql-managed-instance for your Managed Instance?