Hi,
Please be informed that there is no such built-in GPO to totally delete the users folders in Windows system.
We suggest that you could try using Mandatory User Profiles or roaming user profile in your RDS environment. Mandatory user profile will create a new user profile each time when a user logs into the session host and delete it when logging off, while roaming user profile will remove the user profile to the file share sever when a user logs off and again load it back when the user logs in next time.
For your reference:
Create mandatory user profiles
https://learn.microsoft.com/en-us/windows/client-management/mandatory-user-profile
Deploying Roaming User Profiles
https://learn.microsoft.com/en-us/windows-server/storage/folder-redirection/deploy-roaming-user-profiles
If you would not like both of them and want to delete the users folders totally, you may need to search for third party tools or write script to achieve it.
Thanks,
Eleven
----------
If the Answer is helpful, please click "Accept Answer" and upvote it. Thanks.