Instead of using a custom docker container, you can deploy your static React site onto the built-in PHP image.
I set up an example using GitHub Actions to build and deploy an Angular app to App Service Linux (repo here). In the repo, I’m deploying the Angular app onto the built-in PHP image. This correctly serves the Angular app as static content. We are working on a proper static site image on Linux. In the mean-time the workaround is to use the PHP image.
The webapps-deploy action is just using the zip-deploy API under the hood to place the built Angular files under /home/site/wwwroot/. So if you are using another CI/CD provider, you just need to build the files first, then deploy using zip-deploy. Azure DevOps has a task for this.
You may also want to consider hosting your static site onto Azure Storage.
... BTW you should use the blockquote to surround your Dockerfile snippets. Otherwise your comments show up as headers in the post.