CfApi: How to implement Streaming Mode?

WebDAV 106

I want to implement the Cloud Files drive with streaming mode support. I guess this allows building a cloud drive with higher security - no data will be kept on the client-side on disk and all file content and folder content will be automatically deleted.

When creating a drive I set the StreamingAllowed flag:

storageInfo.HydrationPolicy = StorageProviderHydrationPolicy.Progressive;  
storageInfo.HydrationPolicyModifier = StorageProviderHydrationPolicyModifier.StreamingAllowed | StorageProviderHydrationPolicyModifier.AutoDehydrationAllowed;  
storageInfo.PopulationPolicy = StorageProviderPopulationPolicy.Full;

The description of the streaming mode says:

This modifier does not allow the platform to store any data on disk returned by a sync provider.

and also:

grants the platform the permission to not store any data returned by a sync provider on local disks.

My folders content is successfully listed and file content is properly hydrated when I open a file. The folder content is listed on every interaction with Windows Explorer, making a lot of folder listing requests, but I am ok with it as soon as it brings high security.

However, the content of the hydrated files stays on disk and is never deleted, even after machine reboot and when my app is not running and callbacks are not attached. The user can access it for example via a recent files list.

Should I somehow dehydrate files myself or the platform will delete files' content and listed folders automatically at some point?

@Rita Han - MSFT

Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-11-22T02:13:30.133+00:00

Could you please show a minimal, reproducible sample without private information? Or Can you reproduce using CloudMirror sample?
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-11-22T02:28:46.417+00:00

Have you first set up your own cloud provider by registering the sync root directory using the CfRegisterSyncRoot API configuring it with the CF_HYDRATION_POLICY_MODIFIER_STREAMING_ALLOWED policy?
WebDAV 106 Reputation points

2021-11-22T03:27:16.833+00:00

Yes, I use CF_HYDRATION_POLICY_MODIFIER_STREAMING_ALLOWED when registering sync root.
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-11-22T05:29:15.9+00:00

Have you CfUnregisterSyncRoot?
WebDAV 106 Reputation points

2021-11-22T11:26:25.92+00:00

No, I do not call CfUnregisterSyncRoot() function. I simulate the scenario in which the application exited with an error, without unregistering the sync root.

As soon as the documentation says about streaming mode:

This modifier does not allow the platform to store any data on disk returned by a sync provider.

I expect that all downloaded data in hydrated placeholders are deleted when the app exits or al least when the machine is rebooted.

But this is not the case, the file content stays in placeholders after the machine is rebooted.
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-11-23T05:12:28.093+00:00

The CF_HYDRATION_POLICY_MODIFIER_STREAMING_ALLOWED Policy is associated with SyncRoot.
And What happened if CfUnregisterSyncRoot? Perhaps you should CfUnregisterSyncRoot in error handler under unexpected condition.
WebDAV 106 Reputation points

2021-11-23T13:38:32.03+00:00

Even though I can not rely on CfUnregisterSyncRoot() function call (because it may not be called in some cases, for example in case of power failure or other conditions under which the program ends without calling the error handler) I have tested the the CfUnregisterSyncRoot() function call on exit.

The CfUnregisterSyncRoot() behaves as documented - all hydrated files are converted to regular files and stay in the file system on disk.

So the issue is:

Even though the CF_HYDRATION_POLICY_MODIFIER_STREAMING_ALLOWED is specified during registration, the content of the files is persisted on disk.

How do I develop a drive that will not keep data on disk? My aim is to develop a high-security drive that keeps data in memory only and will not persist any data to disk where it may be accessed after the access to the drive is revoked? Also after the app shutdown (regardless power failure, or any other reasons) no data should be accessible for the user.
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-11-24T01:43:04.8+00:00

OK. Could you please show a minimal, reproducible sample?
Yaroslav Chyzh 1 Reputation point

2021-11-25T19:20:22.227+00:00

I have created a fork of the CloudMirror with a single line change:

https://github.com/dobriyeeh/Windows-classic-samples/commit/914de6ce401f7e3a3b836c8af64104ace8cff74e

I run the sample, and than open the file, so it is hydrated.
Exit sample with Ctrl-C

After app exits the hydrated placeholder file is converted back into regular file and remains in the file system.

The expected behaviour: the file should not exist in file system after app exits.

According to StreamingAllowed documentation (https://learn.microsoft.com/en-us/uwp/api/windows.storage.provider.storageproviderhydrationpolicymodifier): This modifier does not allow the platform to store any data on disk returned by a sync provider.
WebDAV 106 Reputation points

2021-12-07T20:16:16.283+00:00

@Xiaopo Yang - MSFT were you able to look at the code and reproduce the issue?
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-12-08T02:27:37.577+00:00

Yes. But Is the condition exiting the sample with Ctrl-C after opening the file suitable for StorageProviderHydrationPolicyModifier::StreamingAllowed ?
WebDAV 106 Reputation points

2021-12-08T02:57:52.22+00:00

I am surprised that you are asking me this question. The documentation does not say anything how to exit the app in the streaming mode as far as I know.

But it clearly says that the data should not be on disk.

And placeholders are obviously converted to regular files and folders when you unregister sync root. It easily reproducible. Which contradicts the streaming mode class reference documentation.
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2021-12-08T07:08:14.16+00:00

OK, I thought I‘ll escalate it.
WebDAV 106 Reputation points

2022-06-22T05:13:22.777+00:00

Any update on this? We still need a drive that does not store any data on the client-side on disk. Or somehow reliably deletes it (even on app crush).
Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2022-06-22T05:18:11.163+00:00

I'm waiting for a response.

1 answer

Umang Mital 1 Reputation point

2023-01-30T14:36:33.7566667+00:00

Hi, I am also facing the same issue, while trying to implement similar scenario. We want to maintain our own caching mechanism for extra prefetching and managing cache size, so we don't want any data to be cache by file provider, it can request whatever it needs, and we will provide the data (either from cache or from cloud), so I though using Streaming_allowed policy will help, but I am also not able to get it working.

Could you please a example of how to use this hydration policy ?
Please sign in to rate this answer.
Umang Mital 1 Reputation point

2023-01-30T14:37:42.3833333+00:00

@Xiaopo Yang - MSFT

Xiaopo Yang - MSFT 11,501 Reputation points Microsoft Vendor

2023-01-31T06:54:06.52+00:00

I haven't gotten any response, but you can open an incident via 'Contact us' tab at link below so that our engineer can work with you closely: https://developer.microsoft.com/en-us/windows/support/ and please choose the 'Technical Support - Coding/Debugging' for Windows SDK for this issue. In-addition, if the support engineer determines that the issue is the result of a bug the service request will be a no-charge case and you won't be charged.
Sign in to comment