The 100% disk usage issue

Question

Hi every body!

I am writing a program of kind "one to many files" (create many signed/crypted block files from one single input file).I am regularly facing the 100%-disk-usage issue as shown in the performance tab of the task manager.
The program sticks and the system becomes unresponsive (any unrelated program aside too..)
As far I can see with the performance monitor, the small block files that I created stay "on air" many times after I closed them, somehow "delegated" to the 'SYSTEM' task (pid 4), which I assume to be the action of the system write cache (?)
The problem is that 'SYSTEM' seems having no limit in accumulating them, in appearance, not taking into account the free memory still available in the system, while at some point, one would flush some cache entries above a reasonable limitto recover free memory (and to avoid keeping not-written data in memory for long!).. (?) Based on the report of the typeperf tool ("Logical disk(_Total)\Disk time percent", (which can report crazy values such as 2500!),I regularly stop the IO requests of my program and spawn the powershell cmdlet Write-VolumeCache to flush the cache, which does not require Administrator privileges..
Some times, it works as expected: the disk activity drops down after few seconds, and my program gently keeps going ahead.. Some other times, It does not do anything.. It returns after a long delay, with the disk still at 100% and typeperf counter still very high..It may stuck very long as is.. In that case, surprisingly, going to deep sleep mode and awaking after few minutes can solve the problem (??).But not every time (and, anyway, that is not a possible work around!)
Does anyone have suggestions to fix that issue ? Small files are created with Win32 default options. I tried using 'CACHE_THRU' options (that is: FILE_FLAG_NO_BUFFERING|FILE_FLAG_WRITE_THROUGH), and the issue seems not occurring anymore, but performances become then unacceptable. What I need is a reasonable compromise between convenient performances with the help of the disk cache and responsiveness/reliability..

Thanks in advance - bye

Olivier

Answer 1

Hi, I'm Elise, and I'd be happy to help with your issue.

For this type of issue I would recommended posting in the Q&A forum, as this is the intended audience for these type of queries:

https://learn.microsoft.com/answers/

Kind Regards,

Elise