Tablet PC Pen and Ink

Article
03/02/2007

Sam George
Microsoft Corporation

June 2002

Applies to:
Microsoft® Windows XP® Tablet PC Edition

Summary: Programmatically delete, select and highlight ink. Also learn what to consider when designing an application that accepts input from a Pen. (20 printed pages)

Overview
Collecting Ink: InkCollector, InkOverlay
Changing the Appearance of Ink
Two Ways to Highlight
Erasing and Selecting with the InkOverlay
Implementing Custom Erasing and Selection with the InkCollector
Handling the Inverted End of a Pen
Conclusion

Overview

The Microsoft® Windows® XP Tablet PC Edition operating system makes collecting, manipulating, and recognizing digital ink simple. This article examines the Tablet PC ink collection objects and describes how to implement highlighting, erasing and selection of ink as well as how to handle pens that have an inverted end.

Collecting Ink: InkCollector, InkOverlay

In the Tablet PC object model, the two primary objects used to collect ink are the InkCollector and the InkOverlay. The InkCollector object is used to capture ink input from any available Tablet PC device and uses a very efficient event sink to render this input in real time. The InkCollector object can collect ink on any HWND and using it is as simple as creating an instance of the InkCollector, specifying an HWND from which to collect ink on, and then enabling the InkCollector.

The InkCollector object is useful for applications that wish to add ink collection, but have existing models for selection and deletion. For applications that do not have existing models for selection and deletion, the InkOverlay object exposes all of the ink collection abilities for the InkCollector and adds support for selecting and erasing ink.

Creating instances of and enabling the InkCollector and InkOverlay objects is simple:

Creating an Instance of InkOverlay in C++

// Headers for Tablet PC Automation interfaces
#include <atlbase.h>
#include <msInkaut.h>
#include <msInkaut_i.c>

CComPtr<IInkOverlay>      spIInkOverlay;
CComPtr<IInkCollector>      spIInkCollector;

// Instance and enable an InkOverlay.
HRESULT hr;
hr = spIInkOverlay.CoCreateInstance(CLSID_InkOverlay);
hr = spIInkOverlay->put_hWnd((long)hwnd);
hr = spIInkOverlay->put_Enabled(VARIANT_TRUE);

// uncomment the following line to instance and enable an InkCollector
// hr = spIInkCollector.CoCreateInstance(CLSID_InkCollector);
// hr = spIInkCollector->put_hWnd((long)hwnd);
// hr = spIInkCollector->put_Enabled(VARIANT_TRUE);

Creating an Instance of InkOverlay in C#

using System;
using Microsoft.Ink;

// Instance and enable an InkOverlay.
InkOverlay inkOverlay = new InkOverlay();
inkOverlay.Handle = this.Handle;
inkOverlay.Enabled = true;

// uncomment the following line to instance and enable an InkCollector
// InkCollector inkCollector = new InkCollector();
// inkCollector.Handle = this.Handle;
// inkCollector.Enabled = true;

Changing the Appearance of Ink

In the Tablet PC object model, the appearance of ink is controlled by manipulating the DrawingAttributes object. The DrawingAttributes associated with ink strokes can be changed at any time. Because the DrawingAttributes object is common to both the InkCollector and InkOverlay, manipulating ink in each object is identical. As an example, the following code changes the default DrawingAttributes that each Tablet cursor will automatically use when drawing to be 100 ink units (1 ink unit = .01 mm) wide, anti-aliased and red in color:

Changing Ink DrawingAttributes in C++

// Headers for Tablet PC Automation interfaces
#include <atlbase.h>
#include <msInkaut.h>
#include <msInkaut_i.c>

// Modify default drawing attributes
CComPtr<IInkDrawingAttributes> spIInkDrawAttrs = NULL;
hr = spIInkOverlay->get_DefaultDrawingAttributes(&spIInkDrawAttrs);
if (SUCCEEDED(hr))
{
    // Modify the width and transparency
    spIInkDrawAttrs->put_Width((float)100);
    spIInkDrawAttrs->put_AntiAliased(VARIANT_TRUE);

    COLORREF color = RGB(255,0,0);
    spIInkDrawAttrs->put_Color(color);      
    
    // Set the new drawing attributes
    spIInkOverlay->putref_DefaultDrawingAttributes(spIInkDrawAttrs);
}

Changing Ink DrawingAttributes in C#

using Microsoft.Ink;

// Set the new drawing attributes
inkOverlay.DefaultDrawingAttributes.Width = 100F;
inkOverlay.DefaultDrawingAttributes.AntiAliasd = true;
inkOverlay.DefaultDrawingAttributes.Color = Color.Red;

Two Ways to Highlight

Applications use the DrawingAttributes object to give ink the appearance of a highlighter. There are really two ways to do this. The first is to simply make the ink transparent by modifying the Transparency property of the DrawingAttributes object, the second way is to use subtractive rendering by modifying the RasterOperation property of the DrawingAttributes object. Set forth below are examples of the code to perform each of these functions and examples of the resulting appearance of the ink.

Highlighting by Setting Transparency in C++

// Modify default drawing attributes
CComPtr<IInkDrawingAttributes> spIInkDrawAttrs = NULL;
hr = spIInkOverlay->get_DefaultDrawingAttributes(&spIInkDrawAttrs);
if (SUCCEEDED(hr))
{
    // Change the color to yellow
    COLORREF color = RGB(255,255,0);
    spIInkDrawAttrs->put_Color(color);      
    
    // Set the transparency
    spIInkDrawAttrs->put_Transparency(175);

    spIInkOverlay->putref_DefaultDrawingAttributes(spIInkDrawAttrs);
}

Highlighting by Setting Transparency in C#

using Microsoft.Ink;

// Set the new drawing attributes
inkOverlay.DefaultDrawingAttributes.Color = Color.Yellow;
inkOverlay.DefaultDrawingAttributes.Transparency = 175;

Figure 1. Resulting ink appearance with transparency set in C#

In the image above, the highlighter was passed over the entire sentence once, and then several more times across the word 'is'; this results in ink accumulating over the word 'is' and becoming less transparent.

Another strategy is to use the RasterOperation property use subtractive rendering. Note that when using this strategy, the ink used for highlighting will not be smoothed or anti-aliased, but it does not accumulate. The Microsoft Journal utility uses this strategy for highlighting ink.

Highlighting by Setting RasterOperation in C++

// Modify default drawing attributes
CComPtr<IInkDrawingAttributes> spIInkDrawAttrs = NULL;
hr = spIInkOverlay->get_DefaultDrawingAttributes(&spIInkDrawAttrs);
if (SUCCEEDED(hr))
{
    // Change the color to yellow
    COLORREF color = RGB(255,255,0);
    spIInkDrawAttrs->put_Color(color);      
    
    // Set the RasterOperation to MaskPen
    spIInkDrawAttrs->put_RasterOperation(IRO_MaskPen);

    spIInkOverlay->putref_DefaultDrawingAttributes(spIInkDrawAttrs);
}

Highlighting by Setting RasterOperation in C#

using Microsoft.Ink;

// Set the new drawing attributes
inkOverlay.DefaultDrawingAttributes.Color = Color.Yellow;
inkOverlay.DefaultDrawingAttributes.RasterOperation =
                              RasterOperation.MaskPen

Figure 2. Resulting ink appearance with RasterOperation set in C#

Erasing and Selecting with the InkOverlay

As mentioned previously, the InkOverlay adds the ability to select and delete ink. To do this is very simple; you just disable the InkOverlay, change its EditingMode property and then enable it. The Tablet PC changes the cursor, outlines selected ink, and deletes ink when the eraser cursor is passed over a stroke, and more. The following code demonstrates how to change to selection or erasing mode:

Moving the InkOverlay to Selection or Deletion mode in C++

// disable the overlay and change to delete mode
spIInkOverlay->put_Enabled(VARIANT_FALSE);

spIInkOverlay->put_EditingMode(IOEM_Delete);
// NOTE: to change to selection, just comment out the line
// above and use this:
// spIInkOverlay->put_EditingMode(IOEM_Select);

// specify stroke deleting
spIInkOverlay->put_DeleteMode(IOERM_StrokeDelete);

// re-enable the overlay, we're done!
spIInkOverlay->put_Enabled(VARIANT_TRUE);

Moving the InkOverlay to Selection or Deletion mode in C#

inkOverlay.Enabled = false;
inkOverlay.EditingMode = InkOverlayEditingMode.Delete;

// specify stroke deleting
inkOverlay.DeleteMode = InkOverlayDeleterMode.StrokeDelete;

// NOTE: to change to selection, just comment out the line
// above and use this:
// inkOverlay.EditingMode = InkOverlayEditingMode.Select;

// re-enable the overlay, we're done!
inkOverlay.Enabled = true;

Implementing Custom Erasing and Selection with the InkCollector

Most applications will simply use the InkOverlay's model for selection and deletion; but some applications might already have a selection and/or deletion model. As an example, consider an application that already has a rectangular wire frame selection model, similar to Microsoft Paint:

Figure 3. The Microsoft Paint rectangular wire frame selection tool

Instead of adding a second selection model by using the InkOverlay's selection model, an application might choose to provide the ability to additionally select ink strokes along with whatever else it allows the user to select. Assuming the application knows the device coordinates of the selection rectangle, it can use the following code to determine which strokes to include in the selection:

Determining the Strokes to Select for a Given Rectangle in C++

// assume 'rect' is a Rectangle value type in device coordinates

// get a pointer to the IInkRenderer interface and IInkDisp interface
CComPtr<IInkRenderer> spIInkRenderer;
hr = spIInkOverlay->get_Renderer(&spIInkRenderer);

CComPtr<IInkDisp> spIInkDisp;
hr = spIInkOverlay->get_Ink(&spIInkDisp);

// create two points that represent the upper left and 
// lower right pixel coords
POINT upperLeft = {rect.left, rect.top};
POINT lowerRight = {rect.right - rect.left, rect.bottom - rect.top};

// convert those two points to ink space
HDC hdc = ::GetDC(g_hWnd);
hr = spIInkRenderer->PixelToInkSpace((long)hdc, 
                                     &upperLeft.x, 
                                     &upperLeft.y);

hr = spIInkRenderer->PixelToInkSpace((long)hdc, 
                                     &lowerRight.x, 
                                     &lowerRight.y);

::ReleaseDC(g_hWnd, hdc);

// create an InkRectangle
CComPtr<IInkRectangle> spIInkRectangle;
hr = spIInkRectangle.CoCreateInstance(CLSID_InkRectangle);

//set the bounds of the rectangle
hr = spIInkRectangle->SetRectangle(upperLeft.y, 
                                   upperLeft.x, 
                                   lowerRight.y, 
                                   lowerRight.x);

//perform a hit test
CComPtr<IInkStrokes> spIInkStrokesHitTest;
hr = spIInkDisp->HitTestWithRectangle(spIInkRectangle, 
                                      90.0, 
                                      &spIInkStrokesHitTest);

// spIInkStrokesHitTest now contains a collection of strokes that 
// all have at least 90% of their points in the specified 
ink rectangle.

Determining the Strokes to Select for a Given Rectangle in C#

// assume 'rect' is a RECT structure in device coordinates

// create two points that represent the upper left and 
// lower right pixel coords
Point upperLeft = new Point(rect.X, rect.Y);
Point lowerRight = new Point(rect.Width - rect.X, 
                             rect.Height - rect.Y};

// convert those two points to ink space
Point[] points = new Point[]{upperLeft, lowerRight};

inkOverlay.Renderer.PixelToInkSpace(inkOverlay.Handle, 
                                    ref points);


// create an Rectangle in ink coordinates
Rectangle inkRectangle = 
     new Rectangle(points[0].X,
          points[0].Y,
          points[1].X - points[0].X,
          points[1].Y - points[0].Y);

//perform a hit test
Strokes strokesHitTest = inkOverlay.Ink.HitTest(inkRectangle, 90F); 

// strokesHitTest now contains a collection of strokes that 
// all have at least 90% of their points in the specified 
ink rectangle.

Now the application can do whatever it wants with the strokes in the selection: copy them to the clipboard, cut them to the clipboard, or delete them.

Similarly, if an application already has an erasing model and would like to just use the Tablet PC to determine which strokes to delete when erasing, it can use the following code to do so (this code assumes that the application is in its erasing mode and that the application knows the current point where the mouse is located)

Determining the Strokes to Delete for a Given Point in C++

// Assume POINT 'pt' is the current location of the cursor

// get a pointer to the IInkRenderer interface and IInkDisp interface
CComPtr<IInkRenderer> spIInkRenderer;
hr = g_pIInkOverlay->get_Renderer(&spIInkRenderer);

CComPtr<IInkDisp> spIInkDisp;
hr = g_pIInkOverlay->get_Ink(&spIInkDisp);

//convert the point to ink space
HDC hdc = ::GetDC(g_hWnd);
hr = spIInkRenderer->PixelToInkSpace((long)hdc, &pt.x, &pt.y);
::ReleaseDC(g_hWnd, hdc);

//perform a hit test
CComPtr<IInkStrokes> spIInkStrokesHitTest;
hr = spIInkDisp->HitTestCircle(pt.x, pt.y, 30.0, 
&spIInkStrokesHitTest);

// spIInkStrokesHitTest now contains a collection of strokes intersect
// within 30 ink units of the point 'pt'

// delete the strokes
hr = spIInkDisp->DeleteStrokes(spIInkStrokesHitTest);

Determining the Strokes to Delete for a Given Point in C#

// Assume point 'pt' is the current location of the cursor
Point pt;   

// Convert the specified point from pixel to ink space coordinates
// NOTE: the 'this' pointer refers to a Windows Forms instance
inkCollector.Renderer.PixelToInkSpace(this.Handle, ref pt);

// Perform a HitTest to find the collection of strokes that are
// intersected by the point (that has now been converted to ink space)
// the second argument specifies the radius, in ink coordinates, 
to search
Strokes strokesHit = inkCollector.Ink.HitTest(pt, 30F);

if (strokesHit.Count > 0)
{
    // Delete all strokes that were hit by the point
    inkCollector.Ink.DeleteStrokes(strokesHit);

    // Repaint the screen to reflect the change
    this.Refresh();
}

Handling the Inverted End of a Pen

Some pens that come with a Tablet PC may have two ends. When pens with two ends are present on a Tablet PC, the application is free to do whatever it wants (usually referred to as the inverted or eraser end), but it is recommended that the application move into its deletion mode – either its own or the InkOverlay's – when the inverted end of the pen is in range of the digitizer. This is done in the 'CursorInRange' event. The following code assumes that the application has already defined a CursorInRange event handler, which is called whenever a pen or mouse enters the area on which the InkCollector or InkOverlay is collecting.

Determining if the Inverted End of a Pen is Present in C++

HRESULT Application::CursorInRange(
        IInkCursor* pIInkCursor, 
        VARIANT_BOOL,
     VARIANT
        )
{
   VARIANT_BOOL inverted;
   HRESULT hr = pIInkCursor->get_Inverted(&inverted);

   if(SUCCEEDED(hr) && inverted == VARIANT_TRUE)
   {
        // disable the overlay and change to delete mode
        spIInkOverlay->put_Enabled(VARIANT_FALSE);

        spIInkOverlay->put_EditingMode(IOEM_Delete);
        // specify stroke deleting
        spIInkOverlay->put_DeleteMode(IOERM_StrokeDelete);

        // re-enable the overlay, we're done!
        spIInkOverlay->put_Enabled(VARIANT_TRUE);
   }
}

Determining if the Inverted End of a Pen is Present in C#

private void CursorInRange_Event(object sender,
                          InkCollectorCursorInRangeEventArgs e)
{
     // check to see if the cursor that is in range is inverted
     if(e.Cursor.Inverted)
     {
          inkOverlay.Enabled = false;
          inkOverlay.EditingMode = InkOverlayEditingMode.Delete;

          // specify stroke deleting
          inkOverlay.DeleteMode = InkOverlayDeleterMode.StrokeDelete;

          // re-enable the overlay, we're done!
          inkOverlay.Enabled = true;
     }
}

This will move the InkOverlay into stroke delete EditingMode and will begin deleting ink. Of course, an additional condition should be added to move the pen back to selection or ink EditingMode if the cursor is not inverted.

Conclusion

The Tablet PC API's have been designed to make it easy for developers to add inking capabilities to their applications. This article has presented the basic platform capabilities and design considerations for Tablet PC applications with regard to the pen, ink, highlighting, selection, and erasing. You can use this information to write applications for Windows XP Tablet PC edition quickly and easily.

Share via