Hi Gabor Varga and Christoph , Thanks for sharing your findings. All the detail provided by you on this issue has been shared with the Product team.
They are aware of this issue and confirmed that -
The fix for the recycle bin issue is planned as part of the hotfix 1 patch. In the meantime, we recommend disabling the policy as suggested and keep an eye out for the release.
Mitigation
Disable the RoamRecycleBin setting using the registry or Group Policy.
Resolution
FSLogix 2210 hotfix 1 will have updates to resolve this issue.
I will also post an update here when hotfix is released.