Memory-Mapped Files
A memory-mapped file contains the contents of a file in virtual memory. This mapping between a file and memory space enables an application, including multiple processes, to modify the file by reading and writing directly to the memory. Starting with the .NET Framework version 4, you can use managed code to access memory-mapped files in the same way that native Windows functions access memory-mapped files, as described in Managing Memory-Mapped Files in Win32 in the MSDN Library.
There are two types of memory-mapped files:
Persisted memory-mapped files
Persisted files are memory-mapped files that are associated with a source file on a disk. When the last process has finished working with the file, the data is saved to the source file on the disk. These memory-mapped files are suitable for working with extremely large source files.
Non-persisted memory-mapped files
Non-persisted files are memory-mapped files that are not associated with a file on a disk. When the last process has finished working with the file, the data is lost and the file is reclaimed by garbage collection. These files are suitable for creating shared memory for inter-process communications (IPC).
Processes, Views, and Managing Memory
Memory-mapped files can be shared across multiple processes. Processes can map to the same memory-mapped file by using a common name that is assigned by the process that created the file.
To work with a memory-mapped file, you must create a view of the entire memory-mapped file or a part of it. You can also create multiple views to the same part of the memory-mapped file, thereby creating concurrent memory. For two views to remain concurrent, they have to be created from the same memory-mapped file.
Multiple views may also be necessary if the file is greater than the size of the application’s logical memory space available for memory mapping (2 GB on a 32-bit computer).
There are two types of views: stream access view and random access view. Use stream access views for sequential access to a file; this is recommended for non-persisted files and IPC. Random access views are preferred for working with persisted files.
Memory-mapped files are accessed through the operating system’s memory manager, so the file is automatically partitioned into a number of pages and accessed as needed. You do not have to handle the memory management yourself.
The following illustration shows how multiple processes can have multiple and overlapping views to the same memory-mapped file at the same time.
Multiple and overlapped views to a memory-mapped file
.png "Shows views to a memory-mapped file.")
Programming with Memory-Mapped Files
The following table provides a guide for using memory-mapped file objects and their members.
Task |
Methods or properties to use |
|---|---|
To obtain a MemoryMappedFile object that represents a persisted memory-mapped file from a file on disk. |
MemoryMappedFile.CreateFromFile method. |
To obtain a MemoryMappedFile object that represents a non-persisted memory-mapped file (not associated with a file on disk). |
MemoryMappedFile.CreateNew method. - or - MemoryMappedFile.CreateOrOpen method. |
To obtain a MemoryMappedFile object of an existing memory-mapped file (either persisted or non-persisted). |
MemoryMappedFile.OpenExisting method. |
To obtain a UnmanagedMemoryStream object for a sequentially accessed view to the memory-mapped file. |
|
To obtain a UnmanagedMemoryAccessor object for a random access view to a memory-mapped fie. |
|
To obtain a SafeMemoryMappedViewHandle object to use with unmanaged code. |
MemoryMappedFile.SafeMemoryMappedFileHandle property. - or - MemoryMappedViewAccessor.SafeMemoryMappedViewHandle property. - or - |
To delay allocating memory until a view is created (non-persisted files only). (To determine the current system page size, use the Environment.SystemPageSize property.) |
CreateNew method with the MemoryMappedFileOptions.DelayAllocatePages value. - or - CreateOrOpen methods that have a MemoryMappedFileOptions enumeration as a parameter. |
Security
You can apply access rights when you create a memory-mapped file, by using the following methods that take a MemoryMappedFileAccess enumeration as a parameter:
You can specify access rights for opening an existing memory-mapped file by using the OpenExisting methods that take an MemoryMappedFileRights as a parameter.
In addition, you can include a MemoryMappedFileSecurity object that contains predefined access rules.
To apply new or changed access rules to a memory-mapped file, use the SetAccessControl method. To retrieve access or audit rules from an existing file, use the GetAccessControl method.
Examples
Persisted Memory-Mapped Files
The CreateFromFile methods create a memory-mapped file from an existing file on disk.
The following example creates a memory-mapped view of a part of an extremely large file and manipulates a portion of it.
Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Runtime.InteropServices
Class Program
Sub Main()
Dim offset As Long = &H10000000 ' 256 megabytes
Dim length As Long = &H20000000 ' 512 megabytes
' Create the memory-mapped file.
Using mmf = _
MemoryMappedFile.CreateFromFile("c:\ExtremelyLargeImage.data", _
FileMode.Open,"ImgA")
' Create a random access view, from the 256th megabyte (the offset)
' to the 768th megabyte (the offset plus length).
Using accessor = mmf.CreateViewAccessor(offset, length)
Dim colorSize As Integer = Marshal.SizeOf(GetType(MyColor))
Dim color As MyColor
Dim i As Long = 0
' Make changes to the view.
Do While (i < length)
accessor.Read(i, color)
color.Brighten(10)
accessor.Write(i, color)
Loop
End Using
End Using
End Sub
End Class
Public Structure MyColor
Public Red As Short
Public Green As Short
Public Blue As Short
Public Alpha As Short
' Make the view brighter.
Public Sub Brighten(ByVal value As Short)
Red = CType(Math.Min(Short.MaxValue, _
(CType(Red, Integer) + value)), Short)
Green = CType(Math.Min(Short.MaxValue, _
(CType(Green, Integer) + value)), Short)
Blue = CType(Math.Min(Short.MaxValue, _
(CType(Blue, Integer) + value)), Short)
Alpha = CType(Math.Min(Short.MaxValue, _
(CType(Alpha, Integer) + value)), Short)
End Sub
End Structure
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Runtime.InteropServices;
class Program
{
static void Main(string[] args)
{
long offset = 0x10000000; // 256 megabytes
long length = 0x20000000; // 512 megabytes
// Create the memory-mapped file.
using (var mmf =
MemoryMappedFile.CreateFromFile(@"c:\ExtremelyLargeImage.data",
FileMode.Open,"ImgA"))
{
// Create a random access view, from the 256th megabyte (the offset)
// to the 768th megabyte (the offset plus length).
using (var accessor = mmf.CreateViewAccessor(offset, length))
{
int colorSize = Marshal.SizeOf(typeof(MyColor));
MyColor color;
// Make changes to the view.
for (long i = 0; i < length; i += colorSize)
{
accessor.Read(i, out color);
color.Brighten(10);
accessor.Write(i, ref color);
}
}
}
}
public struct MyColor
{
public short Red;
public short Green;
public short Blue;
public short Alpha;
// Make the view brigher.
public void Brighten(short value)
{
Red = (short)Math.Min(short.MaxValue, (int)Red + value);
Green = (short)Math.Min(short.MaxValue, (int)Green + value);
Blue = (short)Math.Min(short.MaxValue, (int)Blue + value);
Alpha = (short)Math.Min(short.MaxValue, (int)Alpha + value);
}
}
}
The following example opens the same memory-mapped file for another process.
Imports System
Imports System.IO.MemoryMappedFiles
Imports System.Runtime.InteropServices
Class Program
Public Shared Sub Main(ByVal args As String())
' Assumes another process has created the memory-mapped file.
Using mmf = MemoryMappedFile.OpenExisting("ImgA")
Using accessor = mmf.CreateViewAccessor(4000000, 2000000)
Dim colorSize As Integer = Marshal.SizeOf(GetType(MyColor))
Dim color As MyColor
' Make changes to the view.
Dim i As Long = 0
While i < 1500000
accessor.Read(i, color)
color.Brighten(30)
accessor.Write(i, color)
i += colorSize
End While
End Using
End Using
End Sub
End Class
Public Structure MyColor
Public Red As Short
Public Green As Short
Public Blue As Short
Public Alpha As Short
' Make the view brigher.
Public Sub Brighten(ByVal value As Short)
Red = CShort(Math.Min(Short.MaxValue, CInt(Red) + value))
Green = CShort(Math.Min(Short.MaxValue, CInt(Green) + value))
Blue = CShort(Math.Min(Short.MaxValue, CInt(Blue) + value))
Alpha = CShort(Math.Min(Short.MaxValue, CInt(Alpha) + value))
End Sub
End Structure
using System;
using System.IO.MemoryMappedFiles;
using System.Runtime.InteropServices;
class Program
{
static void Main(string[] args)
{
// Assumes another process has created the memory-mapped file.
using (var mmf = MemoryMappedFile.OpenExisting("ImgA"))
{
using (var accessor = mmf.CreateViewAccessor(4000000, 2000000))
{
int colorSize = Marshal.SizeOf(typeof(MyColor));
MyColor color;
// Make changes to the view.
for (long i = 0; i < 1500000; i += colorSize)
{
accessor.Read(i, out color);
color.Brighten(20);
accessor.Write(i, ref color);
}
}
}
}
}
public struct MyColor
{
public short Red;
public short Green;
public short Blue;
public short Alpha;
// Make the view brigher.
public void Brighten(short value)
{
Red = (short)Math.Min(short.MaxValue, (int)Red + value);
Green = (short)Math.Min(short.MaxValue, (int)Green + value);
Blue = (short)Math.Min(short.MaxValue, (int)Blue + value);
Alpha = (short)Math.Min(short.MaxValue, (int)Alpha + value);
}
}
Non-Persisted Memory-Mapped Files
The CreateNew and CreateOrOpen methods create a memory-mapped file that is not mapped to an existing file on disk.
The following example consists of three separate processes (console applications) that write Boolean values to a memory-mapped file. The following sequence of actions occur:
Process A creates the memory-mapped file and writes a value to it.
Process B opens the memory-mapped file and writes a value to it.
Process C opens the memory-mapped file and writes a value to it.
Process A reads and displays the values from the memory-mapped file.
After Process A is finished with the memory-mapped file, the file is immediately reclaimed by garbage collection.
To run this example, do the following:
Compile the applications and open three Command Prompt windows.
In the first Command Prompt window, run Process A.
In the second Command Prompt window, run Process B.
Return to Process A and press ENTER.
In the third Command Prompt window, run Process C.
Return to Process A and press ENTER.
The output of Process A is as follows:
Start Process B and press ENTER to continue.
Start Process C and press ENTER to continue.
Process A says: True
Process B says: False
Process C says: True
Process A
Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Threading
Module Module1
' Process A:
Sub Main()
Using mmf As MemoryMappedFile = MemoryMappedFile.CreateNew("testmap", 10000)
Dim mutexCreated As Boolean
Dim mTex As Mutex = New Mutex(True, "testmapmutex", mutexCreated)
Using Stream As MemoryMappedViewStream = mmf.CreateViewStream()
Dim writer As BinaryWriter = New BinaryWriter(Stream)
writer.Write(1)
End Using
mTex.ReleaseMutex()
Console.WriteLine("Start Process B and press ENTER to continue.")
Console.ReadLine()
Console.WriteLine("Start Process C and press ENTER to continue.")
Console.ReadLine()
mTex.WaitOne()
Using Stream As MemoryMappedViewStream = mmf.CreateViewStream()
Dim reader As BinaryReader = New BinaryReader(Stream)
Console.WriteLine("Process A says: {0}", reader.ReadBoolean())
Console.WriteLine("Process B says: {0}", reader.ReadBoolean())
Console.WriteLine("Process C says: {0}", reader.ReadBoolean())
End Using
mTex.ReleaseMutex()
End Using
End Sub
End Module
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;
class Program
{
// Process A:
static void Main(string[] args)
{
using (MemoryMappedFile mmf = MemoryMappedFile.CreateNew("testmap", 10000))
{
bool mutexCreated;
Mutex mutex = new Mutex(true, "testmapmutex", out mutexCreated);
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryWriter writer = new BinaryWriter(stream);
writer.Write(1);
}
mutex.ReleaseMutex();
Console.WriteLine("Start Process B and press ENTER to continue.");
Console.ReadLine();
Console.WriteLine("Start Process C and press ENTER to continue.");
Console.ReadLine();
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream())
{
BinaryReader reader = new BinaryReader(stream);
Console.WriteLine("Process A says: {0}", reader.ReadBoolean());
Console.WriteLine("Process B says: {0}", reader.ReadBoolean());
Console.WriteLine("Process C says: {0}", reader.ReadBoolean());
}
mutex.ReleaseMutex();
}
}
}
Process B
Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Threading
Module Module1
' Process B:
Sub Main()
Try
Using mmf As MemoryMappedFile = MemoryMappedFile.OpenExisting("testmap")
Dim mTex As Mutex = Mutex.OpenExisting("testmapmutex")
mTex.WaitOne()
Using Stream As MemoryMappedViewStream = mmf.CreateViewStream(1, 0)
Dim writer As BinaryWriter = New BinaryWriter(Stream)
writer.Write(0)
End Using
mTex.ReleaseMutex()
End Using
Catch noFile As FileNotFoundException
Console.WriteLine("Memory-mapped file does not exist. Run Process A first." & vbCrLf & noFile.Message)
End Try
End Sub
End Module
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;
class Program
{
// Process B:
static void Main(string[] args)
{
try
{
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
{
Mutex mutex = Mutex.OpenExisting("testmapmutex");
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream(1, 0))
{
BinaryWriter writer = new BinaryWriter(stream);
writer.Write(0);
}
mutex.ReleaseMutex();
}
}
catch (FileNotFoundException)
{
Console.WriteLine("Memory-mapped file does not exist. Run Process A first.");
}
}
}
Process C
Imports System.IO
Imports System.IO.MemoryMappedFiles
Imports System.Threading
Module Module1
' Process C:
Sub Main()
Try
Using mmf As MemoryMappedFile = MemoryMappedFile.OpenExisting("testmap")
Dim mTex As Mutex = Mutex.OpenExisting("testmapmutex")
mTex.WaitOne()
Using Stream As MemoryMappedViewStream = mmf.CreateViewStream(2, 0)
Dim writer As BinaryWriter = New BinaryWriter(Stream)
writer.Write(1)
End Using
mTex.ReleaseMutex()
End Using
Catch noFile As FileNotFoundException
Console.WriteLine("Memory-mapped file does not exist. Run Process A first, then B." & vbCrLf & noFile.Message)
End Try
End Sub
End Module
using System;
using System.IO;
using System.IO.MemoryMappedFiles;
using System.Threading;
class Program
{
// Process C:
static void Main(string[] args)
{
try
{
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("testmap"))
{
Mutex mutex = Mutex.OpenExisting("testmapmutex");
mutex.WaitOne();
using (MemoryMappedViewStream stream = mmf.CreateViewStream(2, 0))
{
BinaryWriter writer = new BinaryWriter(stream);
writer.Write(1);
}
mutex.ReleaseMutex();
}
}
catch (FileNotFoundException)
{
Console.WriteLine("Memory-mapped file does not exist. Run Process A first, then B.");
}
}
}