.NET 4.8 & .NET 6.0 StreamWriter.ReadLine(string) not catched by TaskManager hard drive usage

Question

Hello,

I saw a strange thing on 3 different computers (running Windows 10 or 11 Pro)
During benchmark on CSV with CsvHelper I saw that disk usage wasn't changing on reading data.

the code has been tested with both .NET 4.8 & .NET 6.

The Part I write data on disk and I see on the task manager a peak.
The Part II read data and I don't see anything changing on Read speed.
It could hide data theft or at least unwanted access

Anyone see the same problem?

using System.Diagnostics;  
  
string fileName = Path.Combine(Environment.ExpandEnvironmentVariables(@"%TEMP%"), @"Test.txt");  
Console.WriteLine("QuickTest .NET 6.0");  
Stopwatch stopwatch = Stopwatch.StartNew();  
  
// Part I - Generate CSV file  
Console.WriteLine("Generate CSV file");  
Console.WriteLine("Press any key when ready");  
Console.ReadKey();  
  
stopwatch = Stopwatch.StartNew();  
using (StreamWriter writer = new StreamWriter(fileName))  
{  
    for (int i = 0; i < 5000000; i++)  
    {  
        writer.WriteLine("On the other hand, we denounce with righteous indignation and dislike men who are so beguiled and demoralized by the charms of pleasure of the moment, so blinded by desire, that they cannot foresee the pain and trouble that are bound to ensue; and equal blame belongs to those who fail in their duty through weakness of will, which is the same as saying through shrinking from toil and pain. These cases are perfectly simple and easy to distinguish. In a free hour, when our power of choice is untrammelled and when nothing prevents our being able to do what we like best, every pleasure is to be welcomed and every pain avoided. But in certain circumstances and owing to the claims of duty or the obligations of business it will frequently occur that pleasures have to be repudiated and annoyances accepted. The wise man therefore always holds in these matters to this principle of selection: he rejects pleasures to secure other greater pleasures, or else he endures pains to avoid worse pains.");  
    }  
}  
Console.WriteLine($@"Elapsed time : {stopwatch.ElapsedMilliseconds / 1000} s");  
  
// Part II - Read data  
stopwatch = Stopwatch.StartNew();  
Console.WriteLine("Load CSV file");  
Console.WriteLine("Press any key when ready");  
Console.ReadKey();  
  
var result = File.ReadLines(fileName);  
stopwatch.Stop();  
Console.WriteLine($@"Lines : {result.Count()}");  
Console.WriteLine($@"Elapsed time : {stopwatch.ElapsedMilliseconds / 1000} s");  
Console.WriteLine("Press any key to exit");  
Console.ReadKey();  

I also tried this powershell script from MotoX80. (I just copied the test.txt generated file from %TEMP% to C:\Temp). The read speed is still not showing anything.

while ($true) {$a = Get-Content c:\Temp\test.txt}

As you can see on this picture : the memory used by powershell script increase (all will be released when closing powershell command line). But the read value is still 0.

Vincent

Answer 1

As the file disk blocks were cached during the write, no actual disk reads were required.

Monitor logical reads rather than physical

Answer 2

This file is near 5 GB, it's a big size for cache ?
I have the same symptoms with both HDD & SSD.
I put the PART I in comment and run the program again (so the test file has been generated nearly 2 hours ago).

=> Same result.

Answer 3

I noticed the memory was high so I restarted the system and reran the test. This time the read did not register in task manager. The read was 42% faster using the bench mark in the code. I assume the read was from cache not the physical disk.

QuickTest .NET 6.0
Generate CSV file
Press any key when ready
Load CSV file
Press any key when ready
Lines : 5000000
Elapsed time : 7 s
Press any key to exit

Next, I opened a bunch apps to get the memory up to 70%. This time the test shows the physicals disk read in task manager.

Edit: there is a bug in your timer code as it includes the the time it takes before any key is pressed.

Answer 4

I have always watched the page file delta counter to determine when a given process was doing file I/O that was satisfied by mapping (via page fault) from the system cache and not the result of physical I/O to the disk.

Here I have a PS script that is in an infinite loop reading a file.

Monitor that counter for your test program. It might be easier to see if you add a loop that reads the file each time you press a key.

You can also use the rammap utility to review how much memory is allocated to mapped files. The File Details tab should provide more info on your test.txt file.

https://learn.microsoft.com/en-us/sysinternals/downloads/rammap