It’s almost two years now since people have heard about new revolution in building context-aware applications – Windows 7 & it’s context-aware APIs.

As you know, in Windows 7 Microsoft shipped a platform known as “Windows 7 Sensor Platform” and set of APIs for location awareness. Together the platform is known as “Windows 7 Sensor & Location Platform”. You can read more about the platform, progress of work on it in team’s blog here:  http://blogs.msdn.com/b/sensorslocation/

Location remains one of the most critical assets of context-aware computing area for next decades. The biggest problem of location is that until it’ll become ubiquitous and cheap it can’t be a game changer and bring context-aware computing vision to live.

To make location awareness ubiquitous, we need to have a variety of mechanisms to get location data for different situations:

1. GPS (for outdoors)
2. WiFi triangulation (when you are inside)
3. Cell Towers (when you are on a mobile phone device or use Mobile Broadband)
4. IP location detection (it’s usually about 25 km error)

Unfortunately until Windows 7 appeared getting GPS data wasn’t cheap and ubiquitous itself because:

• everyone had to implement NMEA compliant drivers;
• drivers work via COM with GPS device;
• only one app can talk to GPS at time;
• no unified API to work with GPS devices manufactured by different companies.

This problem was solved in Windows 7 by:

• building one common mechanism to deal with any NMEA-compliant GPS device (eliminating need to support each device separately) – Sensor Platform with support of location sensors
• building one common mechanism for any app to deal with location data – “Location Platform API”
• making these APIs to be accessible by any amount of apps
• making these APIs available in COM format / .NET 3.5 (via open source library) / .NET 4 (System.Devices.Location)

Let’s suppose we build a new machine that we want to be context-aware (which means it will know something about physical environment as well). Let’s suppose we’ll start with location as one of main parameters of physical environment. What we will need?

• GPS device
• Drivers, compatible with Windows 7 Location Platform: GPS | WiFi | IP

GPS Driver

In my case, I’ve used Gobi 2000 card (known as HP Mobile Broadband module if you get it for your HP laptop) as GPS source. Unfortunately, Qualcomm (who is building these Gobi 2000 cards) doesn’t provide GPS drivers for Windows 7 Location Platform. In case you have any of u-blox GPS devices, you can download a separate driver package that will turn your laptop into location-aware device.

But what if you don’t have u-blox GPS device, but a generic one (like Gobi 2000-based in my case)? In order to resolve this issue, you can use this awesome generic GPS driver:

Turbo GPSDirect NMEA Sensor Driver for Windows 7

It has both x86 and x64 implementations, and a tool that will enable you to configure your sensor driver for your GPS based on it’s COM port and speed.

WiFi & IP Location Driver

Unfortunately, there is no generic WiFi location driver. The reason is simple – location detection works by mapping lat/long coordinates to WiFi networks SSIDs in a centralized fashion. This is what Skyhook Wireless was doing to enable location APIs in first generation of iPhone. There are several providers of such service, including Google.

Also Skyhook has program to enable 3rd party developers to build location-aware apps using Skyhook’s data, this program doesn’t support Windows 7 Location Platform. Google doesn’t support Windows 7 Location Platform, either.

But there is a “Geosense for Windows” which uses Google’s services to provide you the information about your location using WiFi & IP only.

Geosense for Windows - Sensor Driver for Windows 7 

It has both x86 and x64 implementations, as well.

Finally, to test your location-aware laptop, you need some apps to work with, right?

There are at least two such apps:

• Bing Maps 3D-based “Windows 7 Navigation App” (written by me about year and half ago)
• Location-enabled Google Maps-based client (written by team behind Geosense for Windows)

There is also a simple example of location-aware service-based gadget, “Weather Gadget” that is part of Windows 7 and shows weather in your location:

Of course, there are lots of apps using location data on phones today, but now that we have laptops with GPS inside, Geosense for Windows, and know how to use Location data in our apps (via Windows 7 Location Platform), it’s a new and important step in life of context-aware computing apps developers:

Location, previously used mostly in vertical solutions, is now ready to make your computer location-aware.

Let’s go ahead and develop some location-aware applications!

Sorry for disappearing for a such long time. Quick updates:

December 2008 - March 2009 - Together with Alexander Popov (MSTLab) and Vladimir Borisov created a prototype of augmented reality-based campus navigation tool

May 2009 - Met with Mary Czerwinski in person to discuss opportunitites for collaboration

September 2009 - Presented my project of digital work environment, "Project Universe", on Microsoft's internal event, "Innovation Think Tank III - Demo Fest", got a lot of positive feedback from several Windows, OCS UX folks as well as from Incubation Community experts (these guys are high-level managemers & individual contributors who focus on incubating new efforts at Microsoft)

March 2010 - Together with Mary Czerwinski & Lisa Woods from MSR Program Management Team organized Context-aware Computing Workshop at Microsoft

June 2010 - Started working as HCI Researcher Intern at VIBE group, MSR Redmond. My research advisor & mentor is Mary Czerwinski, Research Area Manager of VIBE group.

Updates on my experiences are written in Russian in this blog: http://blogs.msdn.com/b/msrinternship_ru/ (see posts by me)

Thanks a lot for attention and support!

I’ve been quite for some time on this blog. The usual day-by-day job (ru) is making most of my time, but the progress on context-awareness research is slowly growing :)

same as year ago.... http://blogs.msdn.com/semantics/archive/2008/09/25/windows-7-context-awareness-on-microsoft-pdc-2008.aspx

anyway today I wanted to say added to Windows 7 and to once-heard promise of WinFS, this time at PDC 2009 we are proud to announce another brick in the puzzle of Microsoft Context-aware Computing vision brought to live:

Microsoft Semantic Engine

Just look at this PDC 2009 Talk:

Microsoft Semantic Engine

See how the Microsoft Semantic Engine brings search, structured query, and analytics together in a simple, unified Search-Discover-Organize usage model. Learn how it addresses the need to have unified access to structured and unstructured enterprise data through easy to use analytical tools. Also learn how to enable business insight to support decision making at all levels within the enterprise. Get an overview of the Semantic Engine, its architecture, the product and its APIs.

• Tags: Servers, SQL Server

Of course this time it should be the most intriguing session. This one will be lead by Naveen Garg, head of Program Management in Microsoft Semantic Engine team (part of SQL Server Org which once incubated WinFS). I can’t say more on this topic until PDC, but be sure, this will be a VERY interesting topic, really!

Expect more news – now from PDC – join us there!

Let me just remind you, Gordon was the one behind the great project of Digital Memories, the “MyLifeBits” project. It is an actual “Memex” project brought to life with Microsoft Research and Gordon Bell as the soul and main author and driver of the project.

The actual reply here is to a recent CNN Interview with Gordon Bell regarding his new book, “Total Recall”, that summarizes the results of experiment Gordon Bell took part in for last few years, the MyLifeBits software project, and all the related research articles, experiments etc. done in this area.

Particular, the question that I would love to comment on is the following one:

CNN: Are you worried about losing your memory?

... Forgetting is not a feature, it's a flaw. I don't think forgetting is an important feature of human memory. I think it's important to be able to remember things accurately.

In this reply, Gordon Bell told about opportunity to remember things accurately. For me, it is a slightly important topic that needs to be fully addressed in a more formal way like an article but for the purpose of this writing and with regard to readers of this blog here are few ideas that briefly comment on this suggestion.

I. In Physics, there is understanding that every process can be described, modeled precisely using Math and Physics formulas. However, it is also known that to be totally accurate everything should be known totally precisely. The opportunity to describe everything precisely in Physics comes down to quantum mechanics and here is a moment that there is an Uncertainty principle postulated by Heisenberg. To be short, the point is you’ll never know either precise position of electron or its precise momentum. Thus being showing something precisely, or, as Gordon Bell names “accurately” is technically impossible. This leads to question, if it is impossible to record some processes (and our life that we record with computer power is about series of different processes, after all) precisely, what options do we have?

II. My answer to this question comes from the so-called “General Semantics” approach, first introduced by Alfred Korzybski in his “Science and Sanity, an Introduction to Non-Aristotelian Systems and General Semantics” in 1948. It is that it is impossible to record things accurately. It is also that what we record always will be based on our opinion of the facts we record. And if use computers then we just build them with our opinions in minds. Thus what is important that a real digital memories system will have it’s own opinion that will be based on opinion of system’s author, or, if system will be intelligent enough, on opinion of its owner.

This answer is based on the following principles of General Semantics (let’s name process an event in this particular discussion):

• The map is not the territory itself – which means that any particular event is not equal to its representation, where territory is an event and map is representation
• Every particular event can have a human representation of it being objectified; but there are different object representations given by different parties;
• Every particular event has n of characteristics, or properties representing it where ; thus each event can have m of objects representing it where   and each object highlights different sets of event properties;
• Every object representing a particular event is an abstraction of this event given by a specific point of view or need because of which some particular event properties are highlighted in this object;
• Every object can have its own object representation; because amount k of event properties showcased in particular object is countable but number n of event properties is uncountable; ; this leads to fact that amount of all possible abstractions of event uncountable.

One of the fundamental principles of people intentions is that if there are several opinions on something the most wise thing we can do is to find common things in these opinions in order to precisely describe that exact something. Thus the most interesting question for me is that actually, a recording system should be able to:

I. Be able to record situation of user from different perspectives/opinions

II. Be able to interact with other digital memories systems used by other people in order to build a better/more correct combined image of the situation

And here comes, that a simple recording system turns into a much more complex context-aware system.

The question comes how based on these ideas we could build a better digital memories system?

When looking on to the past we can see that amount of information available for average person was growing each century but the most dramatic growth was established in the XXI and XX century. Also the less fast growth in previous centuries was easy to handle by average human mind the latest breakthrough was too fast for human brain to adapt. Growth of basic required education as well as growth of information flow around us (books, newspapers, Internet) combined together brought us to a situation when an average person doesn’t process all the information but rather uses the templates to process it automatically. The templates are provided as part of education as well as via aggressive advertising and popular shows on TV, radio, books, Internet.

Another interesting trend is that amount of people dealing with information as their primary job was also dramatically increased in XIX and XX centuries (and is growing even more today). A new job category, knowledge or information worker, appeared on the jobs market.

The third trend appeared as part of information overhead problem is that amount of tasks people process daily also dramatically increased from hundreds to hundreds of thousands of tasks.

Task, or activity, as stated in Activity Theory, is described as dynamic, collaborative, multifaceted, it exists at different levels of granularity, exists across places[1]

It’s the way people solve different problems. The problem of a growing number of tasks leads to a growing complexity in a life of an average person while solving personal and work problems.

The information overhead that humanity is facing now thus consists of inability to efficiently process large information blocks solved by applying templates (which turns off thinking), growing amount of people whose primary job is dealing with information (hence information workers), and growing amount of activities.

[1] “In Supporting Activity in Desktop and Ubiquitous Computing” by Stephen Voida, Elizabeth D. Mynatt, and Blair McIntyre (printed in “Beyond the Desktop Metaphor”, MIT 2008).

Wanted to share with you my excitement on the Project Tuva by MSR. Check it here: http://research.microsoft.com/apps/tools/tuva/index.html

The goal of project is to understand how we can augment video with contextual information.

Let me return back in my memory to about almost 5 years ago when I was a student in my institute and learned physics. Feynman lectures were a big help for me to better understand physics and look at it from a different perspective.

Let me recommend you to look at the project Tuva and you should definitely watch Messenger Series on "Prof. Feynman lecture on Natural Law example - Gravity Law" at Cornell University especially the introduction video is a lot of fan :) For those who ever read Feynman books it will remind how fun, spectacular was this well-known professor.

Enjoy!

This time, let's look onto Microsoft's Information Worker Vision. As you might remember, the Microsoft Center for Information Work team with Russ Burtner, UX Lead Designer, had put together a Productivity Vision back in 2005 for next 5 years.

Long Zheng over his IStartedSomething.com did a great overview of that video... See here: http://www.istartedsomething.com/20070314/microsoft-2010-rtc/

Now, 5 years later, where Microsoft is?

Surprizingly, Microsoft did a lot... Voicemail in Exchange, Unified Communications, Tablet PCs going mainstream... Of course everything here required a lot of direct involvement of Partner network and the results are really interesting...

Read Long Zheng's review: http://www.istartedsomething.com/20090713/reflecting-microsofts-2010-vision-from-2005/

In short, Windows API Code Pack is a project inside Windows SDK Team with goal to provide .NET Developers with managed APIs for Windows. One of the cool new things supported by this Pack are Libraries and Explorer Browser Control.

Do you remember the Communication History concept from Longhorn Prototypes in 2004?

So as I was excited by that prototypes about ugh 5 years I wanted to look if we can Finally do that same stuff in 2009 but with real technologies (I mean it; no WinFS guys).

This is what I was able to achieve with half day of coding (BreadcrumbBar is from VistaBridge Library v1.4):

The screenshot above should show a filtered view of a “Communication” library. The reality is (as far as we do not have any WinFS and there is no Communication Library and messages are not stored in it) that I’ve created a saved search (and it’s really local only as it involves name of my Exchange Mailbox in scope), named it as “Received Messages” and set the scope for my Inbox folder. I did the same for Sent Messages search, too.

The list with filters under “Communication Links” is showing real content of “Filters” folder stored inside the manually created “Communication” library. The messages you see on the screen is my real world copy of Inbox folder of my Exchange mailbox so I cleared personal info from the screenshot. The Explorer Browser Control is used to show saved search results; each time a new item arrives to Inbox in Outlook this search result is automatically updated so actually you can work with your inbox through my app without need to go to Outlook.

Also I didn’t want to focus on Back, Forward buttons as well to Command Bar buttons – hopefully you’ll forgive me :)

The Navigation, Details, Preview, Commands panes are hidden by default (in order to mimicry the Longhorn 2004 Prototype User Experience). I also changed the gray background for the window to a Glass one (it was the goal for Longhorn, too).

So far, it seems that we can make the 2004 Longhorn’s promise a reality also 5 years left from that time and the future is not about just this stuff anymore…

-the Best,

Daniel

I’m sorry for not writing too much last time, too busy with regular job that is you now in this crazy time becoming extremely important :)

So far, I wanted to say that also excuse us and our partners across the globe for no public drivers supporting new Location Platform yet, we are working hard to make these drivers happen, consider to hear more public news in coming months!

This time, however, I want to highlight the community focus on innovation, and, particularly, let me share with you my excitement that first public, community, Location Platform Driver is available!

Dan Griffin, MVP, worked hard to deliver a driver that uses IP address of user’s machine and Live Maps functionality to give you your location. Of course it is not very concrete but kudos to Dan it is the first such driver that gives you opportunity to play with new Location Platform in Windows 7!

You can learn more about it here, and grab here:

http://laptoplojack.codeplex.com/

The coolest thing is that you do not have to use GPS (And thus buy one, connect to your device etc) to get your approximate location and start creating location-aware applications today!

And remember that you can use Windows 7 Navigation Application to see your location on the map :)

Dan, Bravo!

I’ve been keeping silence for last few months. It’s because we are working on a project around context aware systems and augmented reality, bringing ideas together.

Also I can’t disclose more details let me share with you a few links on AR-related projects that were presented @ MSR TechFest 2009 just these days!

Demos:

Sticky Notes in AR

http://research.microsoft.com/en-us/events/techfest2009/demos.aspx (pre-last demo)

unfortunately no video :(

Project Videos:

Core Tools for Augmented Reality Michael Cohen, Principal Researcher from Microsoft Research Redmond explains his TechFest 2009 treasure hunt demonstration using Core Tools for Augmented Reality.

Situated Interaction Researcher Dan Bohus introduces the virtual receptionist, a research project designed to improve human-computer interaction based on conversational behavior.

Hope you’ll find these projects interesting :) And you if you got your ideas on how to marry Augmented Reality and Context Aware systems share your aspirations in this blog :)