Hello @Derek Chen ,
By default Files will not save to one drive unless you configured it that way for AVD. If you want to restrict users from accessing the files, you can control from device redirection policies also you can implement Group policies to control it.
On the other had, you can use FSLogix with Azure file share or NetApp file share. It will map user profile to host user login. You can refer to below document for details
Document: fslogix-containers-azure-files
Refer to: FSlogix explaination
Please "Accept as Answer" and Upvote if any of above helped so that, it can help others in community looking for remediation for the similar issues.