August 2014
Volume 29 Number 8
Windows with C++ : DirectComposition: A Retained-Mode API to Rule Them All
.jpg)
Graphics APIs tend to fall into one of two very different camps. There are the immediate-mode APIs with well-known examples, including Direct2D and Direct3D. Then there are the retained-mode APIs such as Windows Presentation Foundation (WPF) or any XAML or declarative API. Modern browsers offer a clear distinction between the two graphics modes, with Scalable Vector Graphics providing a retained-mode API and the Canvas element providing an immediate-mode API.
Retained-mode assumes the graphics API will retain some representation of the scene, such as a graph or tree of objects, which can then be manipulated over time. This is convenient and simplifies the development of interactive applications. In contrast, an immediate-mode API doesn’t include a built-in scene graph and instead relies on the application to construct the scene using a sequence of drawing commands. This has tremendous performance benefits. An immediate-mode API such as Direct2D will typically buffer vertex data from multiple geometries, coalesce a large number of drawing commands and much more. This is particularly beneficial with a text-rendering pipeline, as glyphs first need to be written to a texture and down-sampled before clear-type filtering is applied and the text makes its way to the render target. This is one of the reasons why many other graphics APIs, and increasingly many third-party applications, now rely on Direct2D and DirectWrite for text rendering.
The choice between immediate-mode and retained-mode traditionally came down to a trade-off between performance and productivity. Developers could pick the Direct2D immediate-mode API for absolute performance or the WPF retained-mode API for productivity or convenience. DirectComposition changes this equation by allowing developers to blend the two far more naturally. It blurs the line between immediate-mode and retained-mode APIs because it provides a retained-mode for graphics, but without imposing any memory or performance overhead. It achieves this feat by focusing on bitmap composition rather than attempting to compete with other graphics APIs. DirectComposition simply provides the visual tree and the composition infrastructure such that bitmaps rendered with other technologies can be easily manipulated and composed together. And unlike WPF, DirectComposition is an integral part of the OS graphics infrastructure and avoids all of the performance and airspace issues that have traditionally plagued WPF applications.
If you’ve read my two previous columns on DirectComposition (msdn.microsoft.com/magazine/dn745861 and msdn.microsoft.com/magazine/dn786854), you should already have a sense of what the composition engine is capable of. Now I want to make that a lot more explicit by showing you how you can use DirectComposition to manipulate visuals drawn with Direct2D in a way that’s very appealing to developers accustomed to retained-mode APIs. I’m going to show you how to create a simple window that presents circles as “objects” that can be created and moved around, with full support for hit testing and changes to Z-order. You can see what this might look like in the example in Figure 1.
.png "Dragging Circles Around")
Figure 1 Dragging Circles Around
Although the circles in Figure 1 are drawn with Direct2D, the application draws a circle only once to a composition surface. This composition surface is then shared among the composition visuals in a visual tree bound to the window. Each visual defines an offset relative to the window at which its content—the composition surface—is positioned and ultimately rendered by the composition engine. The user can create new circles and move them around with a mouse, pen or finger. Every time a circle is selected, it moves to the top of the Z-order so it appears above any other circles in the window. While I certainly don’t need a retained-mode API to achieve such a simple effect, it does serve as a good example of how the DirectComposition API works along with Direct2D to achieve some powerful visual effects. The goal is to build an interactive application whose WM_PAINT handler isn’t responsible for keeping the window’s pixels up-to-date.
I’ll start with a new SampleWindow class that derives from the Window class template I introduced in my previous column. The Window class template just simplifies message dispatching in C++:
struct SampleWindow : Window<SampleWindow>
{
};
As with any modern Windows application, I need to handle dynamic DPI scaling so I’ll add two floating-point members to keep track of the DPI scaling factors for the X and Y axis:
float m_dpiX = 0.0f;
float m_dpiY = 0.0f;
You can initialize these on demand, as I illustrated in my previous column, or within your WM_CREATE message handler. Either way, you need to call the MonitorFromWindow function to determine the monitor that has the largest area intersecting the new window. Then you simply call the GetDpiForMonitor function to retrieve its effective DPI values. I’ve illustrated this a number of times in previous columns and courses so I won’t reiterate it here.
I’ll use a Direct2D ellipse geometry object to describe the circle to be drawn so I can later use this same geometry object for hit testing. While it’s more efficient to draw a D2D1_ELLIPSE structure than a geometry object, the geometry object provides hit testing and the drawing will be retained. I’ll keep track of both the Direct2D factory and the ellipse geometry:
ComPtr<ID2D1Factory2> m_factory;
ComPtr<ID2D1EllipseGeometry> m_geometry;
In my previous column I showed you how to create a Direct2D device object directly with the D2D1CreateDevice function rather than by using a Direct2D factory. This is certainly an acceptable way to continue, but there’s a catch. Direct2D factory resources, while they are device-independent and need not be recreated when device loss occurs, can be used only with Direct2D devices created by the same Direct2D factory. Because I want to create the ellipse geometry up front, I need a Direct2D factory object to create it. I could, perhaps, wait until I’ve created the Direct2D device with the D2D1CreateDevice function and then retrieve the underlying factory with the GetFactory method, and then use that factory object to create the geometry, but that seems rather contrived. Instead, I’ll just create a Direct2D factory and use it to create both the ellipse geometry and the device object as needed. Figure 2 illustrates how to create the Direct2D factory and geometry objects.
Figure 2 Creating the Direct2D Factory and Geometry Objects
void CreateFactoryAndGeometry()
{
D2D1_FACTORY_OPTIONS options = {};
#ifdef _DEBUG
options.debugLevel = D2D1_DEBUG_LEVEL_INFORMATION;
#endif
HR(D2D1CreateFactory(D2D1_FACTORY_TYPE_SINGLE_THREADED,
options,
m_factory.GetAddressOf()));
D2D1_ELLIPSE const ellipse = Ellipse(Point2F(50.0f, 50.0f),
49.0f,
49.0f);
HR(m_factory->CreateEllipseGeometry(ellipse,
m_geometry.GetAddressOf()));
}
The CreateFactoryAndGeometry method can then be called by the SampleWindow’s constructor to prepare these device-independent resources. As you can see, the ellipse is defined around a center point 50 pixels along the X and Y axis, as well as a radius of 49 pixels for both the X and Y radius, making this ellipse into a circle. I’ll be creating a 100x100 composition surface. I’ve chosen a radius of 49 pixels because the default stroke drawn by Direct2D straddles the perimeter and it would otherwise be clipped.
Next up are the device-specific resources. I need a backing Direct3D device, a composition device to commit changes to the visual tree, a composition target to keep the visual tree alive, a root visual that will represent the parent of all of the circle visuals, and a shared composition surface:
ComPtr<ID3D11Device> m_device3D;
ComPtr<IDCompositionDesktopDevice> m_device;
ComPtr<IDCompositionTarget> m_target;
ComPtr<IDCompositionVisual2> m_rootVisual;
ComPtr<IDCompositionSurface> m_surface;
I’ve introduced these various objects in my earlier DirectX articles and, in particular, in my two previous columns on DirectComposition. I also discussed and illustrated how you should handle device creation and loss so I won’t repeat that here. I’ll just call out the CreateDevice2D method that needs to be updated to use the previously created Direct2D factory:
ComPtr<ID2D1Device> CreateDevice2D()
{
ComPtr<IDXGIDevice3> deviceX;
HR(m_device3D.As(&deviceX));
ComPtr<ID2D1Device> device2D;
HR(m_factory->CreateDevice(deviceX.Get(),
device2D.GetAddressOf()));
return device2D;
}
Now I’ll create the shared surface. I need to be careful to use the ComPtr class template’s ReleaseAndGetAddressOf method to ensure the surface can be safely recreated after device loss or due to changes in DPI scaling. I also need to be careful to preserve the logical coordinate system that my application is using while translating the dimensions into physical pixels for the DirectComposition API:
HR(m_device->CreateSurface(
static_cast<unsigned>(LogicalToPhysical(100, m_dpiX)),
static_cast<unsigned>(LogicalToPhysical(100, m_dpiY)),
DXGI_FORMAT_B8G8R8A8_UNORM,
DXGI_ALPHA_MODE_PREMULTIPLIED,
m_surface.ReleaseAndGetAddressOf()));
I can then call the composition surface’s BeginDraw method to receive a Direct2D device context with which to buffer drawing commands:
HR(m_surface->BeginDraw(
nullptr,
__uuidof(dc),
reinterpret_cast<void **>(dc.GetAddressOf()),
&offset));
And then I need to tell Direct2D how to scale any drawing commands:
dc->SetDpi(m_dpiX,
m_dpiY);
And I need to transform the output to the offset provided by DirectComposition:
dc->SetTransform(Matrix3x2F::Translation(PhysicalToLogical(offset.x, m_dpiX),
PhysicalToLogical(offset.y, m_dpiY)));
PhysicalToLogical is a helper function I routinely use for DPI scaling when combining APIs that have differing levels of support for DPI scaling (or none at all). You can see the PhysicalToLogical function and the corresponding LogicalToPhysical function in Figure 3.
Figure 3 Converting Between Logical and Physical Pixels
template <typename T>
static float PhysicalToLogical(T const pixel,
float const dpi)
{
return pixel * 96.0f / dpi;
}
template <typename T>
static float LogicalToPhysical(T const pixel,
float const dpi)
{
return pixel * dpi / 96.0f;
}
Now I can simply draw a blue circle with a solid color brush created for just this purpose:
ComPtr<ID2D1SolidColorBrush> brush;
D2D1_COLOR_F const color = ColorF(0.0f, 0.5f, 1.0f, 0.8f);
HR(dc->CreateSolidColorBrush(color,
brush.GetAddressOf()));
Next, I must clear the render target before filling the ellipse geometry and then stroking or drawing its outline with the modified brush:
dc->Clear();
dc->FillGeometry(m_geometry.Get(),
brush.Get());
brush->SetColor(ColorF(1.0f, 1.0f, 1.0f));
dc->DrawGeometry(m_geometry.Get(),
brush.Get());
Finally, I must call the EndDraw method to indicate the surface is ready for composition:
HR(m_surface->EndDraw());
Now it’s time to create circles. In my previous columns I created only a single root visual, but this application needs to create visuals on demand, so I’ll just wrap that up in a convenient helper method:
ComPtr<IDCompositionVisual2> CreateVisual()
{
ComPtr<IDCompositionVisual2> visual;
HR(m_device->CreateVisual(visual.GetAddressOf()));
return visual;
}
One of the interesting aspects of the DirectComposition API is that it’s effectively a write-only interface to the composition engine. While it retains a visual tree for your window, it doesn’t provide any getters you can use to interrogate the visual tree. Any information, such as a visual’s position or Z-order, must be retained directly by the application. This avoids unnecessary memory overhead and also avoids potential race conditions between the application’s view of the world and the composition engine’s transactional state. So I’ll go ahead and create a Circle structure to keep track of each circle’s position:
struct Circle
{
ComPtr<IDCompositionVisual2> Visual;
float LogicalX = 0.0f;
float LogicalY = 0.0f;
};
The composition visual effectively represents the circle’s setters while the LogicalX and LogicalY fields are the getters. I can set the visual’s position with the IDCompositionVisual2 interface and I can retain and later retrieve its position with the other fields. This is necessary both for hit testing and for restoring the circles after device loss. To avoid these getting out of sync, I’ll simply provide a helper method for updating the visual object based on the logical position. The DirectComposition API has no idea how the content might be positioned and scaled, so I need to make the necessary DPI calculations myself:
void UpdateVisualOffset(float const dpiX,
float const dpiY)
{
HR(Visual->SetOffsetX(LogicalToPhysical(LogicalX, dpiX)));
HR(Visual->SetOffsetY(LogicalToPhysical(LogicalY, dpiY)));
}
I’ll add another helper method for actually setting the circle’s logical offset. This one relies on UpdateVisualOffset to ensure that the Circle structure and the visual object are in sync:
void SetLogicalOffset(float const logicalX,
float const logicalY,
float const dpiX,
float const dpiY)
{
LogicalX = logicalX;
LogicalY = logicalY;
UpdateVisualOffset(dpiX,
dpiY);
}
Finally, as circles are added to the application, I’ll need a simple constructor to initialize the structure, taking ownership of an IDCompositionVisual2 reference:
Circle(ComPtr<IDCompositionVisual2> && visual,
float const logicalX,
float const logicalY,
float const dpiX,
float const dpiY) :
Visual(move(visual))
{
SetLogicalOffset(logicalX,
logicalY,
dpiX,
dpiY);
}
I can now keep track of all the application’s circles with a standard list container:
list<Circle> m_circles;
While I’m here I’ll also add a member to track any selected circle:
Circle * m_selected = nullptr;
float m_mouseX = 0.0f;
float m_mouseY = 0.0f;
The mouse offset will also help to produce natural movement. I’ll get the housekeeping out the way before I look at the actual mouse interaction that will ultimately create the circles and allow me to move them around. The CreateDeviceResources method needs to recreate any visual objects, should device loss occur, based on any previously created circles. It wouldn’t do if the circles disappear. So right after creating or recreating the root visual and the shared surface, I’ll iterate over this list, create new visual objects and reposition them to match the existing state. Figure 4 illustrates how this all comes together using what I’ve already established.
Figure 4 Creating the Device Stack and Visual Tree
void CreateDeviceResources()
{
ASSERT(!IsDeviceCreated());
CreateDevice3D();
ComPtr<ID2D1Device> const device2D = CreateDevice2D();
HR(DCompositionCreateDevice2(
device2D.Get(),
__uuidof(m_device),
reinterpret_cast<void **>(m_device.ReleaseAndGetAddressOf())));
HR(m_device->CreateTargetForHwnd(m_window,
true,
m_target.ReleaseAndGetAddressOf()));
m_rootVisual = CreateVisual();
HR(m_target->SetRoot(m_rootVisual.Get()));
CreateDeviceScaleResources();
for (Circle & circle : m_circles)
{
circle.Visual = CreateVisual();
HR(circle.Visual->SetContent(m_surface.Get()));
HR(m_rootVisual->AddVisual(circle.Visual.Get(), false, nullptr));
circle.UpdateVisualOffset(m_dpiX, m_dpiY);
}
HR(m_device->Commit());
}
The other bit of housekeeping has to do with DPI scaling. The composition surface that contains the circle’s pixels as rendered by Direct2D must be recreated to scale, and the visuals themselves must also be repositioned so their offsets are proportional to one another and to the owning window. The WM_DPICHANGED message handler first recreates the composition surface—with the help of the CreateDeviceScaleResources method—and then updates the content and position for each of the circles:
if (!IsDeviceCreated()) return;
CreateDeviceScaleResources();
for (Circle & circle : m_circles)
{
HR(circle.Visual->SetContent(m_surface.Get()));
circle.UpdateVisualOffset(m_dpiX, m_dpiY);
}
HR(m_device->Commit());
Now I’ll deal with the pointer interaction. I’ll let the user create new circles if the left mouse button is clicked while the Control key is pressed. The WM_LBUTTONDOWN message handler looks something like this:
if (wparam & MK_CONTROL)
{
// Create new circle
}
else
{
// Look for existing circle
}
HR(m_device->Commit());
Assuming a new circle needs to be created, I’ll start by creating a new visual and setting the shared content before adding it as a child of the root visual:
ComPtr<IDCompositionVisual2> visual = CreateVisual();
HR(visual->SetContent(m_surface.Get()));
HR(m_rootVisual->AddVisual(visual.Get(), false, nullptr));
The new visual is added in front of any existing visuals. That’s the AddVisual method’s second parameter at work. If I had set this to true then the new visual would’ve been placed at the back of any existing siblings. Next, I need to add a Circle structure to the list so I can later support hit testing, device loss and DPI scaling:
m_circles.emplace_front(move(visual),
PhysicalToLogical(LOWORD(lparam), m_dpiX) - 50.0f,
PhysicalToLogical(HIWORD(lparam), m_dpiY) - 50.0f,
m_dpiX,
m_dpiY);
I’m careful to place the newly created Circle at the front of the list so I can naturally support hit testing in the same order the visual tree implies. I also initially position the visual so it’s centered on the mouse position. Finally, assuming the user doesn’t immediately release the mouse, I also capture the mouse and keep track of which circle will potentially be moved around:
SetCapture(m_window);
m_selected = &m_circles.front();
m_mouseX = 50.0f;
m_mouseY = 50.0f;
The mouse offset allows me to smoothly drag any circle regardless of where on the circle the mouse pointer initially goes down. Looking for an existing circle is a little more involved. Here, again, I need to manually apply DPI awareness. Fortunately, Direct2D makes this a breeze. First, I need to iterate over the circles in the natural Z-order. Happily, I already placed new circles at the front of the list so this is a simple matter of iterating from beginning to end:
for (auto circle = begin(m_circles); circle != end(m_circles); ++circle)
{
}
I’m not using a range-based for statement because it’s going to be more convenient to actually have iterators handy in this case. Now where are the circles? Well, each circle keeps track of its logical position relative to the top-left corner of the window. The mouse message’s LPARAM also contains the pointer’s physical position relative to the top-left corner of the window. But it’s not enough to translate them to a common coordinate system because the shape I need to hit test for isn’t a simple rectangle. The shape is defined by a geometry object and Direct2D provides the FillContainsPoint method to implement hit testing. The trick is that the geometry object provides only the shape of the circle and not its position. For hit testing to work effectively, I’ll need to first translate the mouse position such that it’s relative to the geometry object. That’s easy enough:
D2D1_POINT_2F const point =
Point2F(LOWORD(lparam) - LogicalToPhysical(circle->LogicalX, m_dpiX),
HIWORD(lparam) - LogicalToPhysical(circle->LogicalY, m_dpiY));
But I’m not quite ready to call the FillContainsPoint method. The other issue is that the geometry object doesn’t know anything about the render target. When I used the geometry object to draw the circle, it was the render target that scaled the geometry to match the DPI values of the target. So I need a way to scale the geometry prior to performing hit testing so it will reflect the size of the circle, corresponding to what the user is actually seeing on screen. Once again, Direct2D comes to the rescue. FillContainsPoint accepts an optional 3x2 matrix to transform the geometry prior to testing whether the given point is contained within the shape. I can simply define a scale transform given the window’s DPI values:
D2D1_MATRIX_3X2_F const transform = Matrix3x2F::Scale(m_dpiX / 96.0f,
m_dpiY / 96.0f);
The FillContainsPoint method will then tell me whether the point is contained within the circle:
BOOL contains = false;
HR(m_geometry->FillContainsPoint(point,
transform,
&contains));
if (contains)
{
// Reorder and select circle
break;
}
If the point is contained within the circle, I need to reorder the composition visuals such that the selected circle’s visual is at the top of the Z-order. I can do so by removing the child visual and adding it to the front of any existing visuals:
HR(m_rootVisual->RemoveVisual(circle->Visual.Get()));
HR(m_rootVisual->AddVisual(circle->Visual.Get(), false, nullptr));
I also need to keep my list up-to-date, by moving the circle to the front of the list:
m_circles.splice(begin(m_circles), m_circles, circle);
I then assume the user wants to drag the circle around:
SetCapture(m_window);
m_selected = &*circle;
m_mouseX = PhysicalToLogical(point.x, m_dpiX);
m_mouseY = PhysicalToLogical(point.y, m_dpiY);
Here, I’m careful to calculate the offset of the mouse position relative to the selected circle. In this way the circle doesn’t visually “snap” to the center of the mouse pointer as it’s dragged, providing seamless movement. Responding to the WM_MOUSEMOVE message allows any selected circle to continue this movement so long as a circle is selected:
if (!m_selected) return;
m_selected->SetLogicalOffset(
PhysicalToLogical(GET_X_LPARAM(lparam), m_dpiX) - m_mouseX,
PhysicalToLogical(GET_Y_LPARAM(lparam), m_dpiY) - m_mouseY,
m_dpiX,
m_dpiY);
HR(m_device->Commit());
The Circle structure’s SetLogicalOffset method updates the logical position maintained by the circle, as well as the physical position of the composition visual. I’m also careful to use the GET_X_LPARAM and GET_Y_LPARAM macros to crack the LPARAM, rather than the usual LOWORD and HIWORD macros. While the position reported by the WM_MOUSEMOVE message is relative to the top-left corner of the window, this will include negative coordinates if the mouse is captured and the circle is dragged above or to the left of the window. As usual, changes to the visual tree must be committed for them to be realized. Any movement comes to an end in the WM_LBUTTONUP message handler by releasing the mouse and resetting the m_selected pointer:
ReleaseCapture();
m_selected = nullptr;
Finally, I’ll conclude with the best part. The most compelling proof that this is indicative of retained-mode graphics is when you consider the WM_PAINT message handler in Figure 5.
Figure 5 A Retained-Mode WM_PAINT Message Handler
void PaintHandler()
{
try
{
if (IsDeviceCreated())
{
HR(m_device3D->GetDeviceRemovedReason());
}
else
{
CreateDeviceResources();
}
VERIFY(ValidateRect(m_window, nullptr));
}
catch (ComException const & e)
{
ReleaseDeviceResources();
}
}
The CreateDeviceResources method creates the device stack up front. As long as nothing goes wrong, no further work is done by the WM_PAINT message handler, other than to validate the window. If device loss is detected, the various catch blocks will release the device and invalidate the window as necessary. The next WM_PAINT message to arrive will again recreate the device resources. In my next column I’ll show you how you can produce visual effects that aren’t directly driven by user input. As the composition engine performs more of the rendering without involving the application, it’s possible that device loss might occur without the application even knowing it. That’s why the GetDeviceRemovedReason method is called. If the composition engine detects device loss it will send the application window a WM_PAINT message purely so it can check for device loss by calling the Direct3D device’s GetDeviceRemovedReason method. Take DirectComposition for a test drive with the accompanying sample project!
Kenny Kerr is a computer programmer based in Canada, as well as an author for Pluralsight and a Microsoft MVP. He blogs at kennykerr.ca and you can follow him on Twitter at twitter.com/kennykerr.
Thanks to the following Microsoft technical experts for reviewing this article: Leonardo Blanco (Leonardo.Blanco@microsoft.com) and James Clarke (James.Clarke@microsoft.com)