Advanced Application restart by design is not intended for overlapped. It should be considered more of a task kill to mitigate any process issue. That's why you're seeing that behavior. It's the same for restarting from the portal as well. However, I do echo @TP 's response above in leveraging App Configuration ConfigureRefresh
method to register changes.
If your configuration is rooted in simple App Settings, one approach to consider slot swapping, Set up staging environments - Azure App Service | Microsoft Learn. The advantage here is you can add/remove your application settings within you dev/test slot, route a small percentage of traffic to the slot to verify your changes and then swap the slots.
Another recommended approach towards maintaining high availability is using a multi-region approach. You can deploy your application to separate regions and place them behind Traffic Manager, Control traffic with Traffic Manager - Azure App Service | Microsoft Learn, where you designate a primary and secondary. Therefore, when you upgrade and restart your app service, traffic manager will redirect to the secondary while the primary is offline and switch back once it becomes available.