@Uhmazing34 -if you are already connecting in code , from the application, and the networking is working, then it sounds like you have a transient fault that may need to be investigated between the app and the database leading to connections getting dropped
On the other part of the question
You can use App Settings to manage the configuration values - this works nicely if you're also using the "slots" feature and you can use a key vault reference from the app settings to read through the password from key vault REF https://learn.microsoft.com/en-us/azure/app-service/app-service-key-vault-references?tabs=azure-cli
MySQL for Flexible Server also supports managed identity - here you would be relying on the token derived from the App service managed identity REF https://learn.microsoft.com/en-us/azure/mysql/single-server/how-to-connect-with-managed-identity#creating-a-mysql-user-for-your-managed-identity
In many cases this would be better than using basic authentication (username + password pair)