@David Thielen Thanks for posting your query on Microsoft Q&A.
Here is an example: Connect to Azure resources with Managed Identity – Storage account example
To keep the dev server using an explicit setting, set the connection string as an environment variable in your Azure App Service configuration. This way, your app can access it without hard coding the value in your code.