My name is Loke, I’m the Sr. Product Manager here at the Robotics team in Microsoft. We are currently in the process of redesigning our website and one of the new areas we’re thinking of creating is a Showcase section where we’ll highlight some of the cool projects that our community is working on with regards to Robotics on Microsoft Technologies, including RDS.

If you are currently working on a project, or using RDS or any other Microsoft technologies for building Robots (both software and hardware), we’d appreciate it if you could send us a short description of what you’re doing, how you’re using the technologies, and even some feedback about the process. We’d love to hear from you.

Please send the details to roboinfo@microsoft.com. Looking forward to hearing about your awesome projects.

Loke

All three winners spent two days with us in the Microsoft Robotics booth at Maker Faire, and it was a blast. Here is a photo of us trying to frantically setup our booth!

And the Winners Are...

Second Prize goes to Dan Sionov and the Plant Sitter team, who won $3,000 for their plant- watering robot. We loved the idea of having an autonomous robot take care of plants. Here’s Dan (on the right) accepting his well-deserved prize from Ira Snyder, the General Manager for Microsoft Robotics. And here’s a video of the Plant Sitter in action.

The first prize winner is the KEMODA team with the elderly assistance robot, which was built by Todd Christell. Healthcare is a huge growth opportunity for robotics, and Todd did a great job demonstrating the possibilities as seen in this video. Here is Todd (on the right) accepting his $5,000 check. Well-deserved Todd!

 And the grand prize goes to … the SmartTripod robot, designed and built by Arthur Wait. His SmartTripod robot really captured our imagination and demonstrated the type of scenarios that can be solved using robotics as demonstrated by this video. Arthur’s use of the Kinect to implement “Cue Zones” is both unique and very innovative. Here is Arthur (on the left) accepting his grand prize of $10,000. Congratulations to Arthur!

We’d like to thank everyone who participated in the Robotics @ Home competition. It was great to see how the participants used the EDDIE platform and Microsoft Robotics Developer Studio to bring a robot to life, performing useful, and in some instances important tasks.

From all of us on the Microsoft Robotics team -- congratulations!!

Habib Heydarian
Microsoft Robotics 

If you are at Maker Faire Bay Area this weekend, come join us at the Microsoft booth and see the robots in person. We will also be sharing all the details about the winners here soon. Stay tuned for more details!

Habib Heydarian

Microsoft Robotics

Since the Beta, our number one focus has been on improving the overall quality and stability of RDS 4. With your feedback, we’ve fixed a number of important issues in the product as well as made improvements to existing samples. Additionally, we updated RDS 4 to work with the release version of the Kinect for Windows SDK.

We are already seeing RDS 4 being utilized to develop some interesting applications. Speaking of which, last fall, we announced the Robotics @ Home competition, and are thrilled with the submissions we’ve received. Today, we’re excited to reveal the competition’s 10 finalists. Visit the Robotics @ Home Finalists page to read more about the finalists, including how they think robotics will change the world and what they would do with the prize winnings. We look forward to announcing the grand prize winner in May of this year.

Also, our own team has been using RDS 4 for a while now and we’ve come up with a few cool and unique applications. Check out the video of the Kinect Follow Me robot which was created by our team.

[View:https://www.youtube.com/watch?v=3drtre2tlcU]
From all of us on the Microsoft Robotics team, we’d like to thank you for your continued support and your feedback on RDS 4. As always, you can use the forums on the Microsoft Robotics website to send us any additional questions and suggestions.

Habib Heydarian

Microsoft Robotics

When my wife and I both worked fulltime, we had to leave our dog Darwin home alone for 7-8 hours every day, so I decided to build a pet companion to keep him company. Primarily, I wanted to build something that would entertain Darwin, but I also thought it would be fun to create something that would allow me to check on him from work.

My vision for a pet companion was actually less of a robot and more of a remotely operated vehicle.  In fact, the only autonomous behavior in the robot is its ability to find and pick up a ball and reload the ball launcher.  Most importantly, I wanted to be able to interact with the remote environment and have fun doing it.  The “fun” aspect was the most important and had a lot of impact on my design decisions.

I started with the Microsoft Robotics reference platform and the Microsoft Robotics Developer Studio software. My process for designing and building the robot was basically trial and error.  I didn’t have the skills to use 3D modeling programs well enough to be effective, so I would embrace an idea and give it a try.  I built early prototypes using “instant gratification” processes and materials.  In fact, the first prototype was built almost entirely of cardboard and hot glue. Once I had a design that seemed to fit and function, I upgraded to chemically welded styrene and aluminum parts.  In some cases, it took several major iterations before I found a design that actually worked.  In fact, while the current treat dispenser seems pretty simple today, it is actually the third major iteration that I built to completion because the first two had jamming issues.

Building a robot is a challenging endeavor, especially for someone like me who has little electronics experience, no mechanical engineering knowledge, and a limited budget.  I would categorize my most common struggles into two areas, first and probably foremost was budget.

Early on in the project, I was actually focusing on getting the robot to work using a BasicX microcontroller and an IP-enabled web cam (because I happened to already have those things lying around). I spent many hours trying to build a pan/tilt for the big bulky web cam and trying to program my BasicX to be a decent servo controller (let alone network server etc.).  Much of this work turned out to be a waste of time compared to the relatively meager cost of good servo controllers and cheap netbook PCs.  One key lesson learned is I should not shy away from spending a few bucks to achieve my vision.

The second major frustration that kept rearing its head was part fabrication.  In many cases, I thought up a design for a part or component, only to be frustrated by the fact that I couldn’t make it work! I spent many hours at the local home improvement, toy and hobby stores trying to find an available product that had what I needed.  This issue definitely killed many of my ideas.  But, learning from my first challenge, I recently ordered a RepRap Mendel 3D printer and am hopeful this will be less of an issue in the future.

Overall, designing and building a pet companion was a fantastic learning experience and was a lot of fun. Now if I can only get the pet companion to walk my dog for me…

This article was written by Jordan Correa, a Test Developer on the Robotics Team. Jordan’s robot has evolved through several iterations with a lot of hand-built hardware and custom software. You can view a video about Jordan’s robot at https://www.youtube.com/watch?v=afvdZ343-vE

Microsoft at Maker Faire: Gadgeteer and Kinect

It was a great experience and I would like to share some “Behind the Scenes” info about what it was like being there.

Maker Fair 2011

This was my first time at Maker Faire, and it was amazing.  If you haven’t been, I highly recommend it.  We had the Robo-Razzi robot roaming taking pictures to the delight of the crowds (Smile for the camera!).  I spent nearly all my time in the Microsoft tent running the Kinect demo for scores of people each day. My arms were sore at the end of each day but it was worth it…  we shared some cool technology with thousands of people and got a blue ribbon award for an “excellent exhibit”. 

The ‘Driving Eddie with Kinect’ demo was a big hit. There were lines to try it much of the day both days. I had to make some last minute adjustments to the code when we first set it up and fine tune it for one of the bots, but other than that, it ran flawlessly for hours and people were just amazed. The only down time was to swap laptop batteries every few hours or to switch to a charged robot toward the end of the day and I got those down-times to under 3 minutes.

I can’t begin to tell you how many giggling kids and fascinated adults absolutely loved it. I got to see people’s faces light up all day long. I was constantly saying “OK… we have a line… times up… who’s next?” I repeatedly heard… “That is just amazing”… “Incredible!”. So many people wanted to know more and loved hearing all about it and many wanted to try it over and over again. I saw Parents take extra time to explain to their very young kids what was going on and make sure they realized how amazing it was. You could tell they saw this as something important that should be understood. No one was disappointed, everyone left with a smile and many took time to learn more about the EDDIE and the software. It was a great lead-in to talking about RDS4, the contest and Microsoft’s vision of advanced software for robots.

Everyone tall enough to see over the monitor into the robot pen could get the hang of how to drive it quickly and it was fascinating to watch them learn to drive and develop their own driving style. The best thing was when people would come and watch the robot moving around in the pen for a while without knowing how it was being controlled… then they would see someone to the side with her hands out rolling back and forth like an airplane and they would put one-and-one together and I could see them just light up with excitement and get in line. Then there were people who got in line without knowing what was going on. They saw people waving in front of a big screen and wanted to try. When they stepped up and I told them they would be controlling that robot over there with their body they couldn’t believe it and when it responded to them the smile was priceless. It pleased men, women, boys, and girls of all ages.

I purposely had some of the demo code showing behind the client app and lots of people asked about it… “Is that the code”, “did you write that”, “what language is that?”, “was it hard to write?”, “how many lines of code?”. I would take time to answer them or direct them to another team member so I could keep the line moving.

There was one young woman who got my usual fifteen second instruction and then immediately was driving it around like a pro… I could tell by the way she stood and the fluid, graceful motion of her arms, and how her wrists and fingers perfectly matched her arm movements that she was a dancer… probably ballet. The robot was gracefully swimming around the pen better than I had ever seen before. I asked her if she was a dancer and she smiled and said she was. I told her that the robot was moving more fluidly and beautifully than it had for anyone before her and she smiled and continued to make it dance and swirl for a few minutes. I was so impressed. I felt like I was in a PBS special about computer human interaction.

Another young man wanted to know if it would track someone standing on their head.  So did I, and then next thing I know he is driving the robot around with his legs much to everyone’s delight.  In that orientation, the Kinect software interpreted his legs as arms and away he went.

Several people asked “How is this useful?” or “What are you hoping to accomplish”. I would explain that this is a technology demonstration intended to spark the imagination and invite the user to experience first hand becoming embodied in a simple robot.  While the specific setup we had there might not have a practical application besides entertainment I think we accomplished that goal. As robots become more common in our world, exploring new ways of interacting with them using powerful user interface devices like Kinect is both fun and exciting…  and I’m pretty sure we sparked an imagination or two.

What would you do with an Eddie robot in your home? Enter for a chance to win $10,000!

• Support for the Kinect for Windows SDK Beta 2
• DSS Log Analyzer
• Structured Logging
• Improved Obstacle Avoidance Service with a Visualization UI
• CCR for Silverlight 4.0, including new Extension Methods
• Additional documentation

This version of RDS requires Windows 7 and Visual Studio 2010 (C# Express or higher). You can get a copy of RDS 4 Beta 2 free of charge from the Download page.

Trevor Taylor
Senior Program Manager
Microsoft Robotics Group

    When presented with that question, most people don’t ask for clarification. They intuitively know what the question means, even children do. And then, after naming one or two examples that invariably include a robot tidying up a house one way or another - people often stumble. Children, with their less constrained imagination, dream up  “Transformers inspired” big brother robots, or ones that would do their homework and play games. My kids were interested in a robot with a magic wand, but let’s not discuss synthetic Harry Potter for now. We’ll come back to him, though. 

“If I had asked people what they wanted, they would have said faster horses.”
― Henry Ford
In virtually all cases, people look at their daily lives, their familiar environment and then try imagining how a robot would fit in. No wonder the robot comes up to be either an unrealistically advanced humanoid or a floor-roving disk about a foot across.

Homework first (no pun intended)

   To talk about a “household robot”, let’s first define the term. For the lack of a better definition – we’ll go with “a device that can intelligently move parts or all of itself around a home and do useful stuff”.

The goal of this definition is not to be academically acceptable, but to help us zoom in on a specific set of qualities without implying implementation. The term “intelligently” in this context merely means “not being continuously controlled by a person, and not getting stuck”. Why do we want it to move at all? Because it’s got to have at least some degree of freedom, reach places and interact with the physical world – otherwise it would not be called a robot. And the more mobile it is – the more ‘robotic’ it feels.  Somehow, we humans view autonomously moving objects as ‘alive’ regardless of how intelligent they are. For instance, most Roombas are given a name. But I do not know a single PC owner who bothers to name their desktop, even though a computer is much ‘smarter’ than the randomly roaming vacuum cleaner.

If we visited a supermarket today and looked for products that fit the above description, we’d end up with devices that can be categorized in 2 main groups:

Toys designed to appear intelligent, though usually quite simple inside.

Floor cleaners (and maybe Mowers) - again simple, but useful single-purpose devices.

That’s pretty much it. Get it? Robots have been doing a lot of ‘useful stuff’ on other planets for decades, in manufacturing, versatile Industrial robots have been a norm long before the PC was born, and the military embraced robots earlier then you might think, so why are they so rare in our homes?

Let’s explore some of the reasons, and maybe that will help us get a glimpse on what to expect going forward.

Reason #1 – Discriminating Environment. Our homes, appliances and tools are optimized to be used by people (hardly a surprise) and people only, i.e. if you had a cat with a human IQ of 120, what ‘useful stuff’ could the poor puss do in an unmodified modern apartment? We can’t expect it to be able to whip up an omelet or do the dishes for example. The best we can hope for – is a friend and companion that we must take care of.

Forget cats. Our homes are tricky places for our own children. How would you like eating with a 2 foot long fork from a dinner table that’s taller than you are? Welcome to the world of a 3 year old. But we love our kids and pets and therefore enhance our living space to make them happy.

Any robot looking to inhabit a human dwelling and be ‘useful’ in the utilitarian sense of that word - faces similar challenges. It would have to resemble at least some of the physical and mental characteristics of a grown person in order to perform even mildly complicated tasks. Just imagine the level of dexterity and contextual awareness a robot would need in order to make a pizza in your kitchen. It would have to find, evaluate and pre-process raw ingredients using tools and appliances designed for human hands, eyes, ears, noses, and even human psyche. Furniture, lids, peelers, wraps, labels, the sink, dishes, pans, stove … Makes me smile. Not going to happen. What’s ironic about this example is that a robot is destined to be a far better cook than the vast majority of humans.

    Let’s drill down on this one. Think about it - taking raw ingredients and turning them into a descent stew is not rocket science, after all, even a complicated recipe is just an algorithm with a few dozen standard steps. A machine can do this. We just have to design an appropriate environment. A robot-friendly kitchen must have different tools, and different infrastructure for accessing water, fridge, stove, pantry, drain, trash, etc. It would not take a humanoid to operate such a kitchen, in fact it would be awkward. An ‘integrated’ robot would make more sense, perhaps resembling some of the semi-stationary industrial machines used today in manufacturing with manipulators capable of reaching any item in ‘robot-accessible’ storage, a variety of ‘end of arm’ tools that one minute can be whisking eggs, another – flipping a pancake.

Me: “Hi Kitchy, we are having six guests today. We’ll need a nice 3 course meal. Indian. Not very spicy, ready to be served by 6:15. A salad too. Make sure it’s not very heavy and at least one dish is gluten free.”

Robot: “Any non-vegetarian dishes?”

Me: “Sure, add one.”

Robot: “I am going to make … (lists the menu)”

Me: “Sounds good, and squeeze some fresh orange juice. Bye.”

Can’t wait for the day I’ll have this phone conversation.

“From a new born to a 5 year old – there is a world of distance. From a 5 year old to me – only a step.”

― Leo Tolstoy

Reason #2 – Social Un-Intelligence. I am no expert in the rich and fragmented field of AI, but it is pretty clear that we have yet to overcome some serious hurdles in the human-machine interaction domain. There is a tremendous amount of context to human communication, and we are just beginning to make baby steps in modeling the mechanics of it. But why are we talking about intelligence? Do robots have to be intelligent at all? Aren't socially unaware robots useful?

Sure they are. We can call those – Blue Collar Robots. Cleaning, cooking, maintenance, security, etc. They take on jobs that people must do, but hate doing, don’t want to do, or simply can’t. It’s safe to bet on a steady increase in the number and variety of such machines. Low hanging fruit that include floor cleaning and mowing are already being tackled by a variety of vendors. A step up in complexity – security and surveillance may come next.

And then there is a White Collar Robot – one that helps us do things we love doing. I am most fascinated by this breed. Satisfying curiosity, being creative, socializing, studying, having fun – those sorts of activities are strictly human and therefore only human-centric robots can actively participate and enhance our experiences. We may change our environment to help blue collar bots excel at their jobs, but we can’t modify our own nature, so a white collar robot has a difficult task of grasping the intricacies of human world. And that’s where the cutting edge of science is still not sharp enough. Cute toys and more toys seem to be the only available game in town. There are, of course many research projects, but widespread commercialization of technology has not started yet.

Note, a White Collar robot alone is neither necessary nor sufficient to bring about a household robot boom. But it could greatly accelerate the process because Reason #1 does not apply here quite as strongly. We would readily forgive a few clumsy moves and lack of utility if those were traits of our delightful companion. We would also be open to modify the environment a bit to make our new family member happy, and that’s exactly what the robot race needs to occupy our homes for good. So, an intelligent human-centric robot is quite important. This brings us to Reason #3.

Reason #3 – A Reference Design. Or rather – an apparent lack thereof. We take the basic concept of a personal computer (screen + keyboard + ‘magic box with wires’) for granted now, but it was a great discovery. It has proven to be incredibly versatile, and through custom software let us engage in fundamentally different activities: have fun, do work, learn and communicate. All this using conceptually the same hardware configuration and open design, which made it easier for new industries (i.e. PC software, peripheral devices) to appear and grow quickly. There seems to be no similarly versatile household robot design yet. As mentioned before, it is more than likely that we’ll see more of specialist bots in the near future, but it raises a question: who is going to manage them? Eventually it is us, of course, but constantly reprogramming a legion of independent machines to fit our volatile calendar – does not scale. We will need a ‘central command center’ of some sort, or … a white collar concierge robot to act as a liaison between its less social hard working brethren and people. What physical, sensorial and communicational characteristics would such a device need in order to do ‘all things robotic’ the same way a PC can do “all things digital”? How intelligent will it have to be? Will it need a screen?  Manipulators?  Will it even be a physical entity or a virtual one living on the home network, or somewhere in the cloud? And will we even call it a robot at that point?

No one knows for sure. The only way to find out is to take a few educated guesses and give it a try. That’s exactly what we did at Microsoft with the MARK design (Mobile Autonomous Robot using Kinect). We believe it’s a great platform for imagining up household robots. Here is one example. Microsoft Robotics Developer Studio 4 supports MARK in the simulator, and RDS services will be available when our partners start selling the hardware. You won’t have to do soldering, debug firmware or write low level control code. All built in devices, including Kinect are supported by RDS, and can be accessed from high level, C# code. As my friend Gershon put it – “a robot should be something you program, not something you solder”.

“One man's magic is another man's engineering.”

― Robert A. Heinlein

    When my 8 year old daughter wished for a robot with a magic wand, and my 5 y/o empathically agreed - I initially dismissed their silliness. Then I thought a bit more, defined a magic wand as “a portable device that takes commands from its holder and changes the state of objects” and realized it was me who was being silly. We all have magic wands, of course. The fact that remote controls are relatively single-purpose is not a function of technological limits or even cost, but of practicality. Sure there are ‘universals’ that can operate just about anything that supports a remote, and can even be pprogrammed for new devices, or ‘learn’, but let’s face it: universal remotes are Swiss Army knives – jacks of all trades, and masters of none. They may be cumbersome to set up and they work reasonably well only if all devices are close to each other. As soon as you have more than one room with remotely controllable electronics – a single controller becomes impractical. A robot does not have a problem with misplacing its ‘remote’ or forgetting ‘which button does what’. It can easily interact with 100s of devices ranging from light switches to … other robots in different locations over different protocols and interfaces. Infrared, radio, network, you get the picture. Robots managing robots on your behalf. How do you like that? And it’s not even far-fetched.
“Magic Wand Robot Edition ©” would make most real wizards envious. For it is a 2-way wand. Not only can it ‘command’ other objects, but ‘hear’ them too. RFID technology is so cheap, that it’s not hard to imagine that most items in your house can be tagged. “Daddy, where is my blue doll?” “Don’t know honey, ask Robbie”.

“I never predict anything, and I never will.”
― Paul Gascoigne

     When will house robots start playing a significant role in our daily lives? It’s a tricky question. First, technology needs to be released. Then, if it survives present day, we adapt to use it, and it adapts back to our new usage patterns, which changes how we interact with it further, etc. In the midst of this infinitely recursive stochastic process, our views on what technology ‘should’ look like is constantly redefined and quickly taken for granted in ways that neither consumers, nor technologists or sci-fi writers can predict. 

      Sorry to disappoint, the household robot invasion is not happening tomorrow. Not only because the technology is not ready, but because our mentality and our surroundings have not been modified yet to welcome robots en masse. If that sounds like a catch-22 – it’s because it is. But do not despair, the initial wave of robots will be designed (or is already designed) to survive the ‘stone age’ environment of the modern man, and that’s where it starts. Remember, the first successful electrical appliances were powered through a light bulb screw adapter ...

   So, what is a household robot after all? It’s a class of machines without which we can’t imagine our daily lives. The majority of them are common, cheap and easily recognized, many are hidden, some are specialized, others  - cool and highly coveted. They handle dirty work, inspire our imagination and do everything in between. Our homes are designed to accommodate them. Indeed, you can’t tell where the home ends and robots begin. We depend on them, partner with them, and in return they empower us.  As of Nov 2011 - they are just beginning to be imagined. Too bad I can’t patent this description.

How it Works

The Kinect uses an infrared (IR) laser to spray out a pseudo-random beam pattern. An IR camera captures an image of the dots that are reflected off objects (as in the picture below) and the electronics inside the Kinect figures out how much the dot pattern has been distorted. The distortion is a measure of distance from the camera. This approach is commonly called Structured Light. All this happens at 30 frames per second – not bad for a cheap consumer device.

As a programmer, the Kinect gives you an array of depth values that correspond to the pixels of the RGB image. This unusual coordinate space, consisting of x and y as pixel coordinates and z as a distance in millimeters, is called the Depth Image Space. All of the distances are measured from a virtual plane passing through the Kinect camera. You can convert the data into conventional (x, y z) in meters but this requires additional processing overhead.

Skeleton Space on the other hand uses conventional (x, y, z) coordinates. A skeleton consists of a set of 20 joints, each with its own 3D coordinates. Detecting gestures is relatively easy. For example, to detect a person waving their arm over their head you just need to compare the height of their wrist to see if it is above their head, and then see if it is moving from side to side.

Comparison to Laser Range Finders

Laser Range Finders, or LIDAR (Light Detection and Ranging) devices, have been on the market for a long time. A LRF works by sending out pulses of infrared laser light and timing the return signal. Therefore they are known as Time of Flight devices.

The German SICK brand of LRFs have long been the workhorses of the research community, but these cost thousands of dollars. More recently, Hokuyo in Japan has been selling a cheaper range of LRFs that are approaching the $1,000 barrier. However, for this amount of money you can buy six Kinect sensors, although you would need enough USB ports to plug them all in and the necessary processing power to make use of all the 3D data.

The primary differences between a LRF and a Kinect are the Field of View (FOV), Maximum Range and Resolution. A conventional LRF has a FOV of 180 degrees, and some are up to 270 degrees. In contrast, the Kinect only has a 57 degree FOV.

LRFs are available in a variety of ranges from as little as 2 meters out to well over 100 meters. The Kinect, because it was designed for use in a living room, has a maximum range of 4 meters.

Even with a very large maximum range, the resolution of a LRF is from millimeters to centimeters (depending on how expensive it is) and the accuracy is constant across its entire range. Distance data from the Kinect varies in accuracy from sub-centimeter up close to as much as a 5cm error at its maximum range. This is a consequence of how it measures distance.

One downside of a LRF is that it operates in 2D, effectively taking a horizontal slice through the environment around the robot. In order to capture 3D data, the LRF must be mounted on a tilt mechanism and tilted up and down or mounted sideways and panned from side to side. This makes capturing 3D data relatively slow.

The Future for Kinect

Currently the Kinect has a limited range of 80cm to 4 meters. This means that the Kinect cannot see objects that are right in front of the robot, so you still need traditional obstacle sensors such as sonar (which is why they are included on the RDS Reference Platform). Next year, when the Kinect for Windows Hardware is released, there will be a new “near mode” that will range from 50cm to 3 meters. Although this helps with detecting nearby obstacles, a robot should still have other sensors for redundancy.

Going beyond next year, Microsoft will continue researching even better Kinect hardware. This means that 3D depth data is now here to stay, so sharpen up your 3D geometry skills and get cracking on applications that take full advantage of these new devices.