Quickstart: DOM gestures and manipulations (HTML)

Article
08/31/2015

You can customize the user experience for some of the basic gestures described in the Windows touch language (such as sliding, rotating, and resizing) through basic Document Object Model (DOM) gesture event handling.

Updates for Windows 8.1: Windows 8.1 introduces a number of updates and improvements to the pointer input APIs. See API changes for Windows 8.1 for more info.

If you're new to developing apps using JavaScript: Have a look through these topics to get familiar with the technologies discussed here.

Create your first app using JavaScript

Roadmap for apps using JavaScript

Learn about events with Quickstart: adding HTML controls and handling events

App features, start to finish:

Explore this functionality in more depth as part of our App features, start to finish series

User interaction, start to finish (HTML)

User interaction customization, start to finish (HTML)

User experience guidelines:

The platform control libraries (HTML and XAML) provide a full user interaction experience, including standard interactions, animated physics effects, and visual feedback. If you don't need customized interaction support, use these built-in controls.

If the platform controls are not sufficient, these user interaction guidelines can help you provide a compelling and immersive interaction experience that is consistent across input modes. These guidelines are primarily focused on touch input, but they are still relevant for touchpad, mouse, keyboard, and stylus input.

Samples: See this functionality in action in our app samples.

User interaction customization, start to finish sample

HTML scrolling, panning and zooming sample

Input: DOM pointer event handling sample

Input: Instantiable gestures sample

Objective: To learn how to listen for, handle, and process basic gestures for translation, rotation, and scaling using input from touch, mouse, pen/stylus interactions and DOM gesture events.

Prerequisites

Review Quickstart: Pointers.

We assume that you can create a basic app using JavaScript that uses the Windows Library for JavaScript template.

To complete this tutorial, you need to:

Install Microsoft Visual Studio.
Get a developer license. For instructions, see Develop using Visual Studio 2013.
Create your first app using JavaScript.
Review Quickstart: Adding WinJS controls and styles to learn about WinJS objects and controls.

Time to complete: 30 minutes.

What are gesture events?

A gesture is the physical act or motion performed on, or by, the input device (one or more fingers on a touch surface, a pen/stylus digitizer, mouse, and so on). These natural interactions are mapped to operations on elements in both the system and your app. For more information, see Gestures, manipulations, and interactions.

Windows relies on a basic set of gestures to interact with and manipulate the UI.

Gesture		Description
Tap		A single contact is detected and lifted immediately. Tapping on an element invokes its primary action.
Press and hold		A single contact is detected and does not move. Press and hold causes detailed info or teaching visuals (for example, a tooltip or context menu) to be displayed without a commitment to an action.
Slide		One or more contacts are detected and move in the same direction. We use slide primarily for panning interactions but it can also be used for moving, drawing, or writing.
Swipe		One or more contacts are detected and move a short distance in the same direction. Swipe to select, command, and move.
Turn		Two or more contacts are detected and rotate in a clockwise or counter-clockwise arc. Turn to rotate.
Pinch		Two or more contacts are detected and move closer together. Pinch to zoom out.
Stretch		Two or more contacts are detected and move farther apart. Stretch to zoom in.
For more info on these gestures and how they relate to the Windows touch language, see Touch interaction design.

You can use gesture detection to extend your app's interaction model and build upon the basic pointer events described in Quickstart: Handling pointer input. In fact, your app will most likely consume gesture events (such as handling taps, panning or moving with slide, and zooming with pinch or stretch) and use raw pointer data to support gesture detection and processing.

Your app can process multiple gestures concurrently (such as zooming and rotating), group pointer contacts to target a specific element (such as associating all contacts with the target of the initial, or primary, contact), and identify the specific elements targeted by a particular gesture or pointer contact.

Important If you implement your own interaction support, keep in mind that users expect an intuitive experience involving direct interaction with the UI elements in your app. We recommend that you model your custom interactions on the platform control libraries (HTML and XAML) to keep things consistent and discoverable. The controls in these libraries provide a full user interaction experience, including standard interactions, animated physics effects, visual feedback, and accessibility. Create custom interactions only if there is a clear, well-defined requirement and basic interactions don't support your scenario.

Create the UI

For this example, we use a rectangle (target) as the target object for pointer input and gesture detection and processing.

The rectangle acts as a basic color mixer. The color of the target changes based on an RGB color selection (red, green, or blue) and the target's angle of rotation as reported through the rotation gesture. (We calculate the red, green, or blue value from the angle of rotation.)

We display details for each pointer and gesture event, plus the current transform matrix applied to the target, within the target object.

This is the HTML for this example.

<html>
<head>
    <meta charset="utf-8" />
    <title>PointerInput</title>

    <!-- WinJS references -->
    <link href="//Microsoft.WinJS.2.0/css/ui-dark.css" rel="stylesheet" />
    <script src="//Microsoft.WinJS.2.0/js/base.js"></script>
    <script src="//Microsoft.WinJS.2.0/js/ui.js"></script>

    <!-- BasicGesture references -->
    <link href="/css/default.css" rel="stylesheet" />
    <script src="/js/default.js"></script>
</head>
<body>
    <div class="TargetContainer" id="targetContainer">
        <div id="colorMixer">
            <input type="radio" name="color" value="R" title="Red" id="red" class="Red" /><label for="red" id="labelRed">Red</label>
            <input type="radio" name="color" value="G" title="Green" id="green" class="Green" /><label for="green" id="labelGreen">Green</label>
            <input type="radio" name="color" value="B" title="Blue" id="blue" class="Blue" /><label for="blue" id="labelBlue">Blue</label>
            <div id="targetLog"></div>
            <div id="eventLog"></div>
        </div>
    </div>
</body>
</html>

This is the Cascading Style Sheets (CSS) for this example.

Note Pointer events don't fire during a pan or zoom interaction. You can disable panning and zooming on a region through the CSS properties msTouchAction, overflow, and -ms-content-zooming.

body {
    overflow: hidden;
    position: relative;
}

div #targetContainer {
/*
Set the width and height properties of the target container to fill the viewport. 
You can set these properties to 100%, but we use 100vw (viewport width) and 100vh (viewport height).
See https://go.microsoft.com/fwlink/?LinkID=301480 for more detail on CSS units supported by Internet Explorer.
*/
    height: 100vw;
    width: 100vh;
    overflow: hidden;
    position: absolute;
}

div #colorMixer {
/*
A manipulation-blocking element is defined as an element that explicitly 
blocks direct manipulation via declarative markup, and instead fires gesture 
events such as MSGestureStart, MSGestureChange, and MSGestureEnd.
*/
    touch-action: none;
    -ms-transform-origin: 0px 0px;
    position: absolute;
    background-color: black;
    border-color: white;
    border-width: thick;
    border-style: solid;
}

div #colorSelector {
    position: relative;
}

div #eventLog {
    -ms-overflow-style:scrollbar;
}

input.Red {
    background-color: rgb(255,0,0);
}

input.Green {
    background-color: rgb(0,255,0);
}

input.Blue {
    background-color: rgb(0,0,255);
}

Listen for pointer and gesture events

This code sets up the color mixer and color selectors, and declares the various event listeners.

In most cases, we recommend that you get pointer info through the event argument of the pointer event handlers in your chosen language framework.

If the event argument doesn't expose the pointer details required by your app, you can get access to extended pointer data from the event argument through the getCurrentPoint and getIntermediatePoints methods or currentPoint and intermediatePoints properties. We recommend using the getCurrentPoint and getIntermediatePoints methods as you can specify the context of the pointer data.

First, we declare global variables, define a data object (colorInfo) to track target state, and initialize both the color mixer (target) and the RGB color selectors.

var _width = 640;
var _height = 640;

var _pointerInfo;
var _targetLog;

var _selectedColor;
var _colorRed, _colorGreen, _colorBlue;

// Color-specific data object.
//   value: The color value (r, g, or b)
//   rotation: The rotation value used to calculate color value.
//   matrix: The transform matrix of the target.
function colorInfo(value, rotation, matrix) {
    this.value = value;
    this.rotation = rotation;
    this.matrix = matrix;
}

function initialize() {
    // Configure the target.
    setTarget();

    // Initialize color tracking.
    setColors();
}

Then, we set up the color mixer, associate a gesture recognizer (msGesture) with the object, and declare the various event listeners.

Tip For this example, there is only one object associated with a gesture recognizer. If your app contains a large number of objects that can be manipulated (such as a jigsaw puzzle), consider dynamically creating a gesture recognizer only when pointer input is detected on a target object. The gesture recognizer can be destroyed when the manipulation is complete (see Input: Instantiable gestures sample for an example of this). To avoid the overhead of creating and destroying gesture recognizers, create a small pool of gesture recognizers at initialization and dynamically assign those as required.

// Configure the interaction target.
function setTarget() {
    //  Set up the target position, size, and transform.
    colorMixer.style.width = _width + "px";
    colorMixer.style.height = _height + "px";
    colorMixer.style.msTransform = (new MSCSSMatrix()).
        translate((window.innerWidth - parseInt(colorMixer.style.width)) / 2.0,
        (window.innerHeight - parseInt(colorMixer.style.height)) / 2.0);

    // Create gesture recognizer.
    var msGesture = new MSGesture();
    msGesture.target = colorMixer;
    colorMixer.gesture = msGesture;
    // Expando property for handling multiple pointer devices.
    colorMixer.gesture.pointerType = null;

    // Expando property to track pointers.
    colorMixer.pointers = [];

    // Declare event handlers.
    colorMixer.addEventListener("pointerdown", onPointerDown, false);
    colorMixer.addEventListener("pointerup", onPointerUp, false);
    colorMixer.addEventListener("pointercancel", onPointerCancel, false);
    colorMixer.addEventListener("lostpointercapture", onLostPointerCapture, false);
    colorMixer.addEventListener("MSGestureChange", onMSGestureChange, false);
    colorMixer.addEventListener("MSGestureTap", onMSGestureTap, false);
    colorMixer.addEventListener("MSGestureEnd", onMSGestureEnd, false);
    colorMixer.addEventListener("MSGestureHold", onMSGestureHold, false);
}

Finally, we initialize the RGB color selectors (with event listeners) and the colorInfo object.

// Initialize values and event listeners for color tracking.
function setColors() {
    var m = new MSCSSMatrix(colorMixer.style.msTransform);
    _colorRed = new colorInfo(0, 0, m);
    _colorGreen = new colorInfo(0, 0, m);
    _colorBlue = new colorInfo(0, 0, m);

    document.getElementById("red").addEventListener("click", onColorChange, false);
    document.getElementById("green").addEventListener("click", onColorChange, false);
    document.getElementById("blue").addEventListener("click", onColorChange, false);
}

// Re-draw target based on transform matrix associated with color selection.
function onColorChange(e) {
    switch (e.target.id) {
        case "red":
            colorMixer.style.msTransform = _colorRed.matrix;
            break;
        case "green":
            colorMixer.style.msTransform = _colorGreen.matrix;
            break;
        case "blue":
            colorMixer.style.msTransform = _colorBlue.matrix;
            break;
    }
    _selectedColor = e.target.id;

    eventLog.innerText = "Color change";
    targetLog.innerText = colorMixer.style.msTransform;
}

Handle the pointer down event

On a pointer down event, we get the selected RGB color and associate the pointer with the gesture recognizer by calling the addPointer method. We track the sequence and pointerType to re-associate the pointer and gesture recognizer, if required.

If no color is selected, we ignore the pointer event.

// Pointer down handler: Attach the pointer to a gesture object.
function onPointerDown(e) {
    // Do not attach pointer if no color selected.
    if (_selectedColor === undefined)
        return;
    _selectedColor = getSelectedColor();

    // Process pointer.
    if (e.target === this) {
        this.style.borderStyle = "double";
        //  Attach first contact and track device.
        if (this.gesture.pointerType === null) {
            this.gesture.addPointer(e.pointerId);
            this.gesture.pointerType = e.pointerType;
        }
            // Attach subsequent contacts from same device.
        else if (e.pointerType === this.gesture.pointerType) {
            this.gesture.addPointer(e.pointerId);
        }
            // New gesture recognizer for new pointer type.
        else {
            var msGesture = new MSGesture();
            msGesture.target = e.target;
            e.target.gesture = msGesture;
            e.target.gesture.pointerType = e.pointerType;
            e.target.gesture.addPointer(e.pointerId);
        }
    }
    eventLog.innerText = "Pointer down";
}

// Get the current color.
function getSelectedColor() {
    var colorSelection = document.getElementsByName("color");
    for (var i = 0; i < colorSelection.length; i++) {
        if (colorSelection[i].checked)
            return colorSelection[i].id;
    }
}

Handle the gesture event

In this code, we process the translation (slide or swipe), rotation, and scale (pinch or stretch) gestures.

// Gesture change handler: Process gestures for translation, rotation, and scaling.
// For this example, we don't track pointer movements.
function onMSGestureChange(e) {
    // Get the target associated with the gesture event.
    var elt = e.gestureObject.target;
    // Get the matrix transform for the target.
    var matrix = new MSCSSMatrix(elt.style.msTransform);

    // Process gestures for translation, rotation, and scaling.
    e.target.style.msTransform = matrix.
        translate(e.offsetX, e.offsetY).
        translate(e.translationX, e.translationY).
        rotate(e.rotation * 180 / Math.PI).
        scale(e.scale).
        translate(-e.offsetX, -e.offsetY);

    // Mix the colors based on rotation value.
    switch (_selectedColor) {
        case "red":
            _colorRed.rotation += ((e.rotation * 180 / Math.PI));
            _colorRed.rotation = _colorRed.rotation % 360;
            targetLog.innerText = _colorRed.rotation.toString();
            if (_colorRed.rotation >= 0)
                _colorRed.value = parseInt(Math.abs(_colorRed.rotation) * (256 / 360));
            else
                _colorRed.value = parseInt((360 - Math.abs(_colorRed.rotation)) * (256 / 360));
            document.getElementById("labelRed").innerText = _colorRed.value.toString();
            _colorRed.matrix = matrix;
            break;
        case "green":
            _colorGreen.rotation += ((e.rotation * 180 / Math.PI));
            _colorGreen.rotation = _colorGreen.rotation % 360;
            targetLog.innerText = _colorGreen.rotation.toString();
            if (_colorGreen.rotation >= 0)
                _colorGreen.value = parseInt(Math.abs(_colorGreen.rotation) * (256 / 360));
            else
                _colorGreen.value = parseInt((360 - Math.abs(_colorGreen.rotation)) * (256 / 360));
            document.getElementById("labelGreen").innerText = _colorGreen.value.toString();
            _colorGreen.matrix = matrix;
            break;
        case "blue":
            _colorBlue.rotation += ((e.rotation * 180 / Math.PI));
            _colorBlue.rotation = _colorBlue.rotation % 360;
            if (_colorBlue.rotation >= 0)
                _colorBlue.value = parseInt(Math.abs(_colorBlue.rotation) * (256 / 360));
            else
                _colorBlue.value = parseInt((360 - Math.abs(_colorBlue.rotation)) * (256 / 360));
            document.getElementById("labelBlue").innerText = _colorBlue.value.toString();
            _colorBlue.matrix = matrix;
            break;
    }
    e.target.style.backgroundColor = "rgb(" + _colorRed.value + ", " + _colorGreen.value + ", " + _colorBlue.value + ")";
    targetLog.innerText = e.target.style.msTransform;
    eventLog.innerText = "Gesture change";
}

Handle other events, as required

In this example, we merely report the other events handled here. A more robust app would provide additional functionality.

// Tap gesture handler: Display event.
// The touch language described in Touch interaction design (https://go.microsoft.com/fwlink/?LinkID=268162),
// specifies that the tap gesture should invoke an elements primary action (such as launching an application 
// or executing a command). 
// The primary action in this sample (color mixing) is performed through the rotation gesture.
function onMSGestureTap(e) {
    eventLog.innerText = "Gesture tap";
}

// Gesture end handler: Display event.
function onMSGestureEnd(e) {
    if (e.target === this) {
        this.style.borderStyle = "solid";
    }
    eventLog.innerText = "Gesture end";
}

// Hold gesture handler: Display event.
function onMSGestureHold(e) {
    eventLog.innerText = "Gesture hold";
}

// Pointer up handler: Display event.
function onPointerUp(e) {
    eventLog.innerText = "Pointer up";
}

// Pointer cancel handler: Display event.
function onPointerCancel(e) {
    eventLog.innerText = "Pointer canceled";
}

// Pointer capture lost handler: Display event.
function onLostPointerCapture(e) {
    eventLog.innerText = "Pointer capture lost";
}

See Related topics at the bottom of this page for links to more complex samples.

Complete example

See DOM gestures and manipulations complete code.

Summary and next steps

In this Quickstart, you learned about handling basic gesture events in Windows Store apps using JavaScript.

Basic gesture recognition, coupled with pointer events, are useful for managing simple interactions such as translation (slide or swipe), rotation, and scaling (pinch or stretch).

For handling more elaborate interactions and providing a fully customized user interaction experience, see Quickstart: Static gestures and Quickstart: Manipulation gestures.

For more info on the Windows 8 touch language, see Touch interaction design.

Developers

Responding to user interaction

Developing Windows Store apps (JavaScript and HTML)

Quickstart: Pointers

Quickstart: Static gestures

Quickstart: Manipulation gestures

Designers

Touch interaction design