Hi @Jeff
Glad to see support was able to resolve your issue and thank you for posting your solution so that others experiencing the same thing can easily reference this! Since the Microsoft Q&A community has a policy that the question author cannot accept their own answer. They can only accept answers by others, I'll repost your solution in case you'd like to Accept the answer.
Issue: You recently noticed that the video files (.MP4) that are hosted in Azure App Service using ASP.NET V4.8 won't play.
Solution: You added a web.config file to the wwwroot folder with this content:
<configuration>
<system.webServer>
<defaultDocument enabled="true">
<files>
<clear />
<add value="index.html"/>
</files>
</defaultDocument>
<staticContent>
<mimeMap fileExtension=".woff2" mimeType="application/x-font-woff" />
<mimeMap fileExtension=".woff" mimeType="application/x-font-woff" />
<mimeMap fileExtension=".json" mimeType="application/json" />
<mimeMap fileExtension=".mp4" mimeType="video/mp4" />
</staticContent>
</system.webServer>
</configuration>