Yes! You need to reestart the webapp. But if you want to minimize impacts to end users, you can adopt a zero downtime architecture using slots or a vertical load balancer. As one endpoint is updated, it is decommissioned on the balancer, redirecting traffic to the second endpoint.
Some options to configure settings:
For ASP.NET and ASP.NET Core developers, setting app settings in App Service are like setting them in <appSettings>
in Web.config or appsettings.json, but the values in App Service override the ones in Web.config or appsettings.json. You can keep development settings (for example, local MySQL password) in Web.config or appsettings.json and production secrets (for example, Azure MySQL database password) safely in App Service. The same code uses your development settings when you debug locally, and it uses your production secrets when deployed to Azure.