Share via

Depth Camera

Glossary Item Box

DSS Interop Samples: Kinect Sensor

Robotics Common: Webcam Sensor

See Also Microsoft Robotics Developer Studio Send feedback on this topic

Depth Camera

The DepthCam Sensor service provides information from a depth camera. At present the only supported depth camera is the Kinect, which means that the DepthCam Sensor service is not really generic.

The DepthCam Sensor service provides the following operations. The Replace operation is used for notifications.

Operation Description
Get Gets the State of the DepthCamSensor service.
HttpGet Gets the State via a HTTP request, usually from a web browser.
HttpQuery Used to request images from a web browser.
Replace Reserved. Do not send this operation request.
Subscribe Subscribe to notifications. Only Replace messages are sent as notifications.

You can Subscribe to the DepthCam Sensor and it will send frames as notifications (Replace messages). Alternatively, you can poll the service using Get.

The state of a Webcam Sensor is represented the following properties.

Name Type Description
DepthImage short[] This is the depth data arranged in scanline order from top to bottom and left to right as a single-dimensional array. Values are in millimeters.
FurtherThanMaxDepthValue short The value of a pixel when it is further than the maximum range.
MaximumRange double Maximum depth value in meters.
MinimumRange double Minimum depth value in meters.
NearerThanMinDepthValue short The value of a pixel when it is nearer than the minimum range.
Pose Pose Defines the position and orientation of the camera. This must be set in an initial state (config) file.
Timestamp DateTime Time when this image was captured.
VisibleImage byte[] The raw visible image data in BGR24 format. Note that the order is not RGB.
DepthImageSize Size The dimensions of the depth image.
FieldOfView double Horizontal Field of View of the camera in Radians.
ProjectionMatrix Matrix A matrix that defines the conversion between real world view space (3D) and camera space (pixel coordinates plus depth).
InverseProjectionMatrix Matrix Inverse of the Projection Matrix, i.e. it converts in the opposite direction.
ImageMode DepthCam-
Reserved for future use.

Note that the state includes both the depth data (as an array of shorts) and the RGB data.

The depth data is 11 bits and is in millimeters measured from the camera as shown in the diagram below:


The depth data includes a player index in the low-order three bits. The depth data is therefore shifted to the left by 3 bits. You must take this into account when reading the depth data.

Depth Image space consists of X and Y coordinates of pixels in the image (normalized from 0 to 1) as well as a depth (Z) value in millimeters. The pixel values do not represent anything in physical space. Note that depth pixels do not align perfectly with RGB pixels for several reasons. The Kinect service provides a way to map depth pixels to color (RGB) pixels.

Skeletons are represented as a set of joints in 3D Skeleton Space. See the Kinect Sensor for more details.

For more information, see the Kinect Services for RDS document in the Documentation folder of your RDS installation.


See Also 

DSS Interop Samples: Kinect Sensor

Robotics Common: Webcam Sensor



© 2012 Microsoft Corporation. All Rights Reserved.