Keep memorystream

Question

Hi,

Sometimes I need to filter some large textfiles and they take a long time to load. So I goolgled a fast way to load e huge textfile into memory. This works and counting the lines of a huge file gets done in seconds once it is in memory. But once the methode is done the data from the file is gone. I would like to have the entire file to stay in memory so I can add, test and adjust my filters till I write the result to a file. The code below I found with google is not mine.

public void UseMemoryStream()
        {
            long testcount = 0;
            byte[] fileContents = File.ReadAllBytes(PathString);
            using (MemoryStream memoryStream = new MemoryStream(fileContents))
            {
                string line;
                using (TextReader textReader = new StreamReader(memoryStream))
                {                   
                    while ((line = textReader.ReadLine()) != null)
                   testcount++;
                }
            }
            TotalLines.Text = testcount.ToString();
}

My question is how to keep this data in memory so I can preform text searches on it.
Thanks in advance for the effort.

Answer 1

I think that you can keep the memoryStream variable to some class member, and also remove the using statement. But maybe it is better to do this:

lines = File.ReadLines(PathString).ToList();

where lines is a member or variable:

List<string> lines;

It will keep the list of lines. You can easily process, insert, remove lines, and use File.WriteAllLines to save the file.

Answer 2

The "using" statement around your MemoryStream disposes it when execution reaches the closing brace } at the end of the using block. This is precisely the reason why you write using, so that it frees the resources used by the MemoryStream. Therefore, if you do not want to release it, but rather you want to keep it in memory, you should not be writing the using statement. Instead, declare the MemoryStream variable wherever you want to keep it, for example, at Form level. And then, when you do not need it any more, call memoryStream.Dispose() to free it.

However, in this particular case, doing all these things is overkill. You can load the whole file into an array of strings using File.ReadAllLines, as mentioned in a previous answer. This is much shorter to write than the memorystream plus the streamreader, and you can easily keep the array of strings in memory as long as you wish. It does not need to be Disposed.