This solution is a good one- to get the VM access to Key Vault without saving credentials there is one extra step. You can add a User-Assigned Managed Identity to all of your VMs in the DTL instance using the policies. Any existing VMs will need to be restarted for this to be applied to them. In contrast, a regular service principal login would still need credentials.
]2
From there, you can use Connect-AzAccount -Identity
to login to Azure Powershell without credentials via that managed identity. You can give that managed identity access to the required information in Key Vault and apply the file share from there.