Hi @Chad Thomsen ,
I am not completely sure why it was creating the file under "C:\app\File" path but as Azure Automation assigns a worker (which is just like a machine) to run each job during runbook execution in the sandbox and as you have mentioned "File" in your New-Item cmdlet so it might be creating the file under C:\ drive.
As per this Azure document, recommendation is to use $env:TEMP folder if you need to create temporary files as part of your runbook logic in the Azure sandbox for runbooks running in Azure.
I have created a runbook with below code in PS runtime 7.1 and file was created under "C:\Windows\System32\File" path as illustrated in below screenshot.
Then I have followed recommendation to use $env:TEMP and have created a runbook with below code and file was created under "C:\Users\Client\Temp" path.
I would recommend you to use $env:TEMP path and see if it helps to avoid the error that you were facing.