Jeff Jinnett: Framework for Designing Applications for Microsoft Surface®

Article
08/04/2010

Microsoft Surface® is a table-top computer that responds to natural hand movements and tag-enabled objects, with four unique capabilities: (a) direct interaction (users can interact with content on-screen by touch and gesture), (b) multi-user experience (the display enables several people to interact in a face-to-face computing experience), (c) multi-touch (the display responds not just to the touch of one finger, as with a typical touch screen, but from dozens of contact points at once) and (d) object recognition (users can place physical objects on the display, such as for the transfer of digital content to mobile devices).

Microsoft itself has developed applications that (1) add the Seadragon[i] image zoom capability to Surface, (2) enable Bluetooth device connectivity[ii], and (3) enable recognition of byte tags and identity tags placed on objects in order to launch applications[iii], among other processes. In addition, Microsoft Surface partners such as Infusion Development Corp., Stimulant, Luminos, Infostrat, nsquared and Vectorform have developed Surface applications that incorporate additional capabilities, such as (1) the use of physical image magnification rings[iv] and on-screen magnification lenses[v] , (2) the ability to explore documents and coordinate calendars[vi], (3) the ability to navigate 3-D data collected from Internet, desktop and Surface data sources[vii], (4) the ability to use stackable objects for 3-D renderings and interaction[viii] (5) the ability to “flip” images from the surface up to a wall monitor with a hand motion[ix], and (6) the ability to interact with fiber optics embedded in objects[x].

Design and development training is now available online for those interested in developing applications for Surface[xi]. Also, a new book is available on a pre-publication basis dealing with multi-touch code development for Surface[xii]. Demos exist illustrating how Surface can be used to create unique user experiences for customers in various industries, such the Razorfish demo for the financial services industry[xiii] and the Infusion Surface application for Barclays[xiv]. A new addition to the Surface suite of tools is the Microsoft Surface Toolkit for Windows Touch Beta, which permits developers to create applications for touch PCs and prepare for the next version of Surface[xv].

In order to better identify situations that are well-suited to the use of Surface and other multi-touch displays, it might be helpful to develop a framework for mapping the capabilities of Surface against business use cases, while taking into account any limiting factors. For example, a customer to a stock brokerage firm could be issued a card that has a Surface identity tag on it. If the customer visits a local office of the stock brokerage firm, the customer could place the card on a Surface display at the brokerage. The tag would identify the customer and ask for the customer to authenticate onscreen by typing in a password. Then the Surface display would connect online to a database containing information about the stocks and bonds owned by the customer. The Surface display would then map that portfolio against various portfolio analysis tools to show the customer how his or her portfolio was performing in real time. The portfolio analysis could be displayed using a 3-D touch-capable visualization tool, such as the Aqumin AlphaVision™ tool[xvi]. One positive factor from a business point of view is that the customer would be encouraged to visit his or her broker at the broker’s local office in order to use the Surface application. One limiting factor could be the reluctance of the customer to have his or her portfolio displayed on a table top display that could be seen by other customers in the vicinity.

Also, if the broker were to assist the customer in the use of the display, the multi-user capability of Surface would be involved. The proposed design framework for Surface applications would need to include a list of potential positive and negative factors to be taken into account in deciding whether the potential Surface application for a particular use case merited development. A subjective and quantitative analysis of successful Surface and other multi-touch user experiences could provide a benchmark against which potential future applications then could be judged.

[i] See https://www.microsoft.com/showcase/en/US/details/2b7260af-873f-499b-acfa-281d84c8d3ba.

[ii] See https://blogs.msdn.com/b/surface/archive/2010/06/15/microsoft-surface-bluetooth-connect-code-sample-now-available-for-download.aspx.

[iii] See https://msdn.microsoft.com/en-us/library/ee804762(Surface.10).aspx, https://msdn.microsoft.com/en-us/library/ee804803(Surface.10).aspx and

[iv] See https://www.microsoft.com/showcase/en/US/details/5cc82ab8-9e1e-4e6a-803d-5ffa6d824f2d.

[v] See https://stimulant.io/wp/index.php/blog/2009/07/microsoft-local-impact-map-surface-edition/.

[vi] See https://blogs.msdn.com/b/surface/archive/2010/07/08/nsquared-presents-their-surface-business-pack.aspx.

[vii] See https://www.microsoft.com/showcase/en/US/details/9cc401dc-5a7c-40bc-8aa7-5787cf52acf2.

[viii] See https://www.pcworld.com/article/194051/luminos_add_third_dimension_to_microsoft_surface.html.

[ix] See https://blogs.msdn.com/b/surface/archive/2010/04/27/experience-the-garibaldi-panorama-on-surface.aspx.

[x] See https://www.microsoft.com/showcase/en/US/details/78929a20-e8d8-48bc-86b2-6af1ee961dd6.

[xi] See https://www.microsoft.com/surface/en/us/Pages/Technical/Training.aspx.

[xii] See Joshua Blake, Multitouch on Windows (Manning Early Access Program), at https://manning.com/blake/.