First Look at the Kinect for Windows SDK
The Kinect for Windows SDK has finally arrived, and we couldn’t wait to get our hands on it!
After downloading, installing, and actually getting a Kinect with a USB connector for our PC’s, we were good to go…
Step 1 – What does the SDK give us?
When we started, we didn’t really know what to expect from the SDK.
What you get are a set of 20 points, each corresponding to a specific joint in the body. These points come in standard ‘x, y, z’ coordinates, with an extra coordinate ‘w’ , which we haven’t got round to figuring out what it is there for just yet.
Here comes problem one. The points that are given are within the range -1 and 1, going from left to right for x, and bottom to top with y. Therefore, you have to first convert these to work with WPF applications so that if fits within the coordinates of a standard page.
This part was just a formality and had a very simple fix with a couple of converters.
One for X:
1: /// <summary>
2: /// Joint converter
3: /// </summary>
4: public class KinectValueToScreenCoOrdinatesConverterX : IValueConverter
5: {
6: /// <summary>
7: /// Converts a value.
8: /// </summary>
9: /// <param name="value">The value produced by the binding source.</param>
10: /// <param name="targetType">The type of the binding target property.</param>
11: /// <param name="parameter">The converter parameter to use.</param>
12: /// <param name="culture">The culture to use in the converter.</param>
13: /// <returns>
14: /// A converted value. If the method returns null, the valid null value is used.
15: /// </returns>
16: public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
17: {
18: double multiplier = Application.Current.MainWindow.Width;
19:
20: double translation = 0;
21: try
22: {
23: if (parameter != null)
24: {
25: translation = double.Parse(parameter as string);
26: }
27:
28: float fl = (float)value;
29: return translation + (multiplier + (fl * multiplier));
30: }
31: catch (InvalidCastException)
32: {
33: return 0;
34: }
35: }
36:
37: /// <summary>
38: /// Converts a value.
39: /// </summary>
40: /// <param name="value">The value that is produced by the binding target.</param>
41: /// <param name="targetType">The type to convert to.</param>
42: /// <param name="parameter">The converter parameter to use.</param>
43: /// <param name="culture">The culture to use in the converter.</param>
44: /// <returns>
45: /// A converted value. If the method returns null, the valid null value is used.
46: /// </returns>
47: public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
48: {
49: throw new NotImplementedException();
50: }
51: }
And one for Y:
1: /// <summary>
2: /// Joint converter Y
3: /// </summary>
4: public class KinectValueToScreenCoOrdinatesConverterY : IValueConverter
5: {
6: /// <summary>
7: /// Converts a value.
8: /// </summary>
9: /// <param name="value">The value produced by the binding source.</param>
10: /// <param name="targetType">The type of the binding target property.</param>
11: /// <param name="parameter">The converter parameter to use.</param>
12: /// <param name="culture">The culture to use in the converter.</param>
13: /// <returns>
14: /// A converted value. If the method returns null, the valid null value is used.
15: /// </returns>
16: public object Convert(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
17: {
18: double multiplier = Application.Current.MainWindow.Height;
19:
20: double translation = 0;
21: try
22: {
23: if (parameter != null)
24: {
25: translation = double.Parse(parameter as string);
26: }
27:
28: float fl = (float)value;
29: return translation + (multiplier + (-fl * multiplier));
30: }
31: catch (InvalidCastException)
32: {
33: return 0;
34: }
35: }
36:
37: /// <summary>
38: /// Converts a value.
39: /// </summary>
40: /// <param name="value">The value that is produced by the binding target.</param>
41: /// <param name="targetType">The type to convert to.</param>
42: /// <param name="parameter">The converter parameter to use.</param>
43: /// <param name="culture">The culture to use in the converter.</param>
44: /// <returns>
45: /// A converted value. If the method returns null, the valid null value is used.
46: /// </returns>
47: public object ConvertBack(object value, Type targetType, object parameter, System.Globalization.CultureInfo culture)
48: {
49: throw new NotImplementedException();
50: }
51: }
A converter is assigned to every part of the skeleton that is displayed on the UI. Since Kinect returns a series of skeletons (each one representing a person) with the points already mapped to each joint, we decided to drive the MVVM pattern directly off of the SkeletonData that Kinect returns.
Step 2 – Putting everything into the MVVM pattern
To simplify things and make it easier to create multiple skeletons on screen at any one time, we decided to use the MVVM pattern. There are four main parts to this:
1. The Kinect connection
2. The Main View Model
3. The Skeleton View Model
4. The UI
Kinect Connection
The Kinect connection is an intermediate class between our view models and the Kinect sensor. It fires Skeleton ready events for each skeleton that is returned and skeleton frame complete events when an entire frame has been processed. As it processes the skeletons in a frame, it keeps track of their ID value so we can track all the skeletons that were in a frame during the Skeleton Frame Complete event.
One of the main parts of this class is the InitializeNui() method:
1: /// <summary>
2: /// Initializes the Kinect sensor.
3: /// </summary>
4: /// <returns>bool value true if the sensor initialised correctly</returns>
5: private bool InitializeNui()
6: {
7: if (this.kinectRunTime == null)
8: {
9: return false;
10: }
11:
12: try
13: {
14: this.kinectRunTime.Initialize(RuntimeOptions.UseDepth | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor);
15: }
16: catch (Exception exception)
17: {
18: Console.WriteLine(exception.ToString());
19: return false;
20: }
21:
22: this.kinectRunTime.VideoStream.Open(ImageStreamType.Video, 2, ImageResolution.Resolution640x480, ImageType.Color);
23:
24: this.kinectRunTime.SkeletonEngine.TransformSmooth = true;
25:
26: var parameters = new TransformSmoothParameters
27: {
28: Smoothing = 0.75f,
29: Correction = 0.0f,
30: Prediction = 0.0f,
31: JitterRadius = 0.05f,
32: MaxDeviationRadius = 0.04f
33: };
34:
35: this.kinectRunTime.SkeletonEngine.SmoothParameters = parameters;
36:
37: return true;
38: }
39: }
This method initialises the Kinect sensor with a set of pre define arguments. The actual initialization of the sensor happens in the line :
1: this.kinectRunTime.Initialize(RuntimeOptions.UseDepth | RuntimeOptions.UseSkeletalTracking | RuntimeOptions.UseColor);
The runtime options specify what you want the Kinect sensor to look for. In our example we were looking for skeleton frames and images from the camera.
Main View Model
The main view model controls the allocation of skeletons to the skeleton view models. It contains a list of Skeleton View models and a dictionary of skeleton view models with the skeleton id as the key. Each time a skeleton ready event is fired in the Kinect connection it looks up the skeleton ID value in the dictionary and assigns the new skeleton data to the skeleton in that view model.
Originally the Main view model was also going to sort out the removal of skeletons that are no longer active. However the Kinect sensor does not always return all of the active skeletons in each of the Skelton frame ready events that the Kinect SDK produce. This means that a simple removal of any skeletons not in the current frame will fail and the view model associated with a skeleton will be continuously deleted and re-created. To get around this we placed a timer in each of the skeleton view models. If the view model is not updated within a set period of time the view model raises an event and the main view model then catches it and deleted the skeleton.
The Skeleton View Model
The skeleton view model is essentially a way of exposing the properties of a SkeletonData object so that the UI can bind to them. When the skeleton property is changed a NotifyPropertyChanged event is raised on all of the exposed properties which updates all the bindings on the UI.
The skeleton view model also controls the Gesture Service which we will discuss in our next blog post. This is again updated when the skeleton property changes.
The full code for the skeleton view model can be found in the source code below:
1: /// <summary>
2: /// The main view model
3: /// </summary>
4: public class SkeletonViewModel : INotifyPropertyChanged
5: {
6: /// <summary>
7: /// The gesture controler for this skeleton
8: /// </summary>
9: private GestureControler gestures = new GestureControler();
10:
11: #region fields
12:
13: /// <summary>
14: /// backing field for the gesture text
15: /// </summary>
16: private string gestureText;
17:
18: /// <summary>
19: /// backing field for the gesture property
20: /// </summary>
21: private bool gestureDetected;
22:
23: /// <summary>
24: /// The skeleton data
25: /// This is used as the backing field for all properties
26: /// </summary>
27: private SkeletonData skeleton;
28:
29: /// <summary>
30: /// backing field for the joint color
31: /// </summary>
32: private Color jointColor;
33:
34: /// <summary>
35: /// Occurs when [delete skeleton].
36: /// </summary>
37: public event EventHandler DeleteSkeleton;
38:
39: /// <summary>
40: /// the timer for the on screen gesture text
41: /// </summary>
42: private DispatcherTimer textTimer = new DispatcherTimer() { Interval = TimeSpan.FromSeconds(3) };
43:
44: #endregion
45:
46: #region constructors
47:
48: /// <summary>
49: /// Initializes a new instance of the <see cref="SkeletonViewModel"/> class.
50: /// </summary>
51: public SkeletonViewModel()
52: {
53: this.ChangeTimer = new DispatcherTimer();
54: this.ChangeTimer.Interval = TimeSpan.FromSeconds(0.5);
55: this.ChangeTimer.Tick += new EventHandler(this.ChangeTimer_Tick);
56: this.textTimer.Tick += new EventHandler(this.TextTimer_Tick);
57: ChangeTimer.Start();
58: DefineGestures();
59: this.gestures.GestureRecognised += new EventHandler<GestureEventArgs>(this.Gestures_GestureRecognised);
60: }
61:
62: #endregion
63:
64: #region events
65:
66: /// <summary>
67: /// Occurs when a property value changes.
68: /// </summary>
69: public event PropertyChangedEventHandler PropertyChanged;
70:
71: #endregion
72:
73: #region properties
74:
75: /// <summary>
76: /// Gets or sets the change timer.
77: /// </summary>
78: /// <value>
79: /// The change timer.
80: /// </value>
81: public DispatcherTimer ChangeTimer
82: {
83: get;
84: set;
85: }
86:
87: /// <summary>
88: /// Gets or sets the gesture text.
89: /// </summary>
90: /// <value>
91: /// The gesture text.
92: /// </value>
93: public string GestureText
94: {
95: get
96: {
97: return this.gestureText;
98: }
99:
100: set
101: {
102: if (this.gestureText != value)
103: {
104: this.gestureText = value;
105: this.NotifyPropertyChanged("GestureText");
106: }
107: }
108: }
109:
110: /// <summary>
111: /// Gets or sets a value indicating whether this <see cref="SkeletonViewModel"/> is waved.
112: /// </summary>
113: /// <value>
114: /// <c>true</c> if waved; otherwise, <c>false</c>.
115: /// </value>
116: public bool GestureDetected
117: {
118: get
119: {
120: return this.gestureDetected;
121: }
122:
123: set
124: {
125: if (this.gestureDetected != value)
126: {
127: this.gestureDetected = value;
128: this.NotifyPropertyChanged("GestureDetected");
129: }
130: }
131: }
132:
133: /// <summary>
134: /// Gets or sets the color of the joint.
135: /// </summary>
136: /// <value>
137: /// The color of the joint.
138: /// </value>
139: public Color JointColor
140: {
141: get
142: {
143: return this.jointColor;
144: }
145:
146: set
147: {
148: if (this.jointColor != value)
149: {
150: this.jointColor = value;
151: this.NotifyPropertyChanged("JointColor");
152: }
153: }
154: }
155:
156: /// <summary>
157: /// Gets or sets the skeleton.
158: /// </summary>
159: /// <value>
160: /// The skeleton.
161: /// </value>
162: public SkeletonData Skeleton
163: {
164: get
165: {
166: return this.skeleton;
167: }
168:
169: set
170: {
171: if (this.skeleton != value)
172: {
173: this.skeleton = value;
174: this.NotifyPropertyChanged("Skeleton");
175: this.NotifyAllChange();
176: ResetTimer();
177: this.gestures.UpdateAllGestures(this.skeleton);
178: }
179: }
180: }
181:
182: #region KinectBodyParts
183:
184: /// <summary>
185: /// Gets the head.
186: /// </summary>
187: public Joint Head
188: {
189: get
190: {
191: if (this.Skeleton != null)
192: {
193: return this.Skeleton.Joints[JointID.Head];
194: }
195: else
196: {
197: return new Joint();
198: }
199: }
200: }
201:
202: /// <summary>
203: /// Gets the left hand.
204: /// </summary>
205: public Joint LeftHand
206: {
207: get
208: {
209: if (this.Skeleton != null)
210: {
211: return this.Skeleton.Joints[JointID.HandLeft];
212: }
213: else
214: {
215: return new Joint();
216: }
217: }
218: }
219:
220: /// <summary>
221: /// Gets the right hand.
222: /// </summary>
223: public Joint RightHand
224: {
225: get
226: {
227: if (this.Skeleton != null)
228: {
229: return this.Skeleton.Joints[JointID.HandRight];
230: }
231: else
232: {
233: return new Joint();
234: }
235: }
236: }
237:
238: /// <summary>
239: /// Gets the left wrist.
240: /// </summary>
241: public Joint LeftWrist
242: {
243: get
244: {
245: if (this.Skeleton != null)
246: {
247: return this.Skeleton.Joints[JointID.WristLeft];
248: }
249: else
250: {
251: return new Joint();
252: }
253: }
254: }
255:
256: /// <summary>
257: /// Gets the right wrist.
258: /// </summary>
259: public Joint RightWrist
260: {
261: get
262: {
263: if (this.Skeleton != null)
264: {
265: return this.Skeleton.Joints[JointID.WristRight];
266: }
267: else
268: {
269: return new Joint();
270: }
271: }
272: }
273:
274: /// <summary>
275: /// Gets the left elbow.
276: /// </summary>
277: public Joint LeftElbow
278: {
279: get
280: {
281: if (this.Skeleton != null)
282: {
283: return this.Skeleton.Joints[JointID.ElbowLeft];
284: }
285: else
286: {
287: return new Joint();
288: }
289: }
290: }
291:
292: /// <summary>
293: /// Gets the right elbow.
294: /// </summary>
295: public Joint RightElbow
296: {
297: get
298: {
299: if (this.Skeleton != null)
300: {
301: return this.Skeleton.Joints[JointID.ElbowRight];
302: }
303: else
304: {
305: return new Joint();
306: }
307: }
308: }
309:
310: /// <summary>
311: /// Gets the shoulder center.
312: /// </summary>
313: public Joint ShoulderCenter
314: {
315: get
316: {
317: if (this.Skeleton != null)
318: {
319: return this.Skeleton.Joints[JointID.ShoulderCenter];
320: }
321: else
322: {
323: return new Joint();
324: }
325: }
326: }
327:
328: /// <summary>
329: /// Gets the spine.
330: /// </summary>
331: public Joint Spine
332: {
333: get
334: {
335: if (this.Skeleton != null)
336: {
337: return this.Skeleton.Joints[JointID.Spine];
338: }
339: else
340: {
341: return new Joint();
342: }
343: }
344: }
345:
346: /// <summary>
347: /// Gets the hip center.
348: /// </summary>
349: public Joint HipCenter
350: {
351: get
352: {
353: if (this.Skeleton != null)
354: {
355: return this.Skeleton.Joints[JointID.HipCenter];
356: }
357: else
358: {
359: return new Joint();
360: }
361: }
362: }
363:
364: /// <summary>
365: /// Gets the left knee.
366: /// </summary>
367: public Joint LeftKnee
368: {
369: get
370: {
371: if (this.Skeleton != null)
372: {
373: return this.Skeleton.Joints[JointID.KneeLeft];
374: }
375: else
376: {
377: return new Joint();
378: }
379: }
380: }
381:
382: /// <summary>
383: /// Gets the right knee.
384: /// </summary>
385: public Joint RightKnee
386: {
387: get
388: {
389: if (this.Skeleton != null)
390: {
391: return this.Skeleton.Joints[JointID.KneeRight];
392: }
393: else
394: {
395: return new Joint();
396: }
397: }
398: }
399:
400: /// <summary>
401: /// Gets the left ankle.
402: /// </summary>
403: public Joint LeftAnkle
404: {
405: get
406: {
407: if (this.Skeleton != null)
408: {
409: return this.Skeleton.Joints[JointID.AnkleLeft];
410: }
411: else
412: {
413: return new Joint();
414: }
415: }
416: }
417:
418: /// <summary>
419: /// Gets the right ankle.
420: /// </summary>
421: public Joint RightAnkle
422: {
423: get
424: {
425: if (this.Skeleton != null)
426: {
427: return this.Skeleton.Joints[JointID.AnkleRight];
428: }
429: else
430: {
431: return new Joint();
432: }
433: }
434: }
435:
436: /// <summary>
437: /// Gets the left foot.
438: /// </summary>
439: public Joint LeftFoot
440: {
441: get
442: {
443: if (this.Skeleton != null)
444: {
445: return this.Skeleton.Joints[JointID.FootLeft];
446: }
447: else
448: {
449: return new Joint();
450: }
451: }
452: }
453:
454: /// <summary>
455: /// Gets the right foot.
456: /// </summary>
457: public Joint RightFoot
458: {
459: get
460: {
461: if (this.Skeleton != null)
462: {
463: return this.Skeleton.Joints[JointID.FootRight];
464: }
465: else
466: {
467: return new Joint();
468: }
469: }
470: }
471:
472: /// <summary>
473: /// Gets the Left Hip
474: /// </summary>
475: public Joint RightHip
476: {
477: get
478: {
479: if (this.Skeleton != null)
480: {
481: return this.Skeleton.Joints[JointID.HipRight];
482: }
483: else
484: {
485: return new Joint();
486: }
487: }
488: }
489:
490: /// <summary>
491: /// Gets the Left Hip
492: /// </summary>
493: public Joint LeftHip
494: {
495: get
496: {
497: if (this.Skeleton != null)
498: {
499: return this.Skeleton.Joints[JointID.HipLeft];
500: }
501: else
502: {
503: return new Joint();
504: }
505: }
506: }
507:
508: /// <summary>
509: /// Gets the Right Shoulder
510: /// </summary>
511: public Joint RightShoulder
512: {
513: get
514: {
515: if (this.Skeleton != null)
516: {
517: return this.Skeleton.Joints[JointID.ShoulderRight];
518: }
519: else
520: {
521: return new Joint();
522: }
523: }
524: }
525:
526: /// <summary>
527: /// Gets the Left Shoulder
528: /// </summary>
529: public Joint LeftShoulder
530: {
531: get
532: {
533: if (this.Skeleton != null)
534: {
535: return this.Skeleton.Joints[JointID.ShoulderLeft];
536: }
537: else
538: {
539: return new Joint();
540: }
541: }
542: }
543: #endregion
544:
545: #endregion
546:
547: #region methods
548:
549: /// <summary>
550: /// Handles the Tick event of the textTimer control.
551: /// </summary>
552: /// <param name="sender">The source of the event.</param>
553: /// <param name="e">The <see cref="System.EventArgs"/> instance containing the event data.</param>
554: private void TextTimer_Tick(object sender, EventArgs e)
555: {
556: this.GestureDetected = false;
557: this.textTimer.Stop();
558: }
559:
560: /// <summary>
561: /// Handles the GestureRecognised event of the Gestures control.
562: /// </summary>
563: /// <param name="sender">The source of the event.</param>
564: /// <param name="e">The <see cref="KinectSkeltonTracker.GestureEventArgs"/> instance containing the event data.</param>
565: private void Gestures_GestureRecognised(object sender, GestureEventArgs e)
566: {
567: if (e.GestureType == GestureType.WaveRight)
568: {
569: this.GestureDetected = true;
570: this.GestureText = "Waved with right hand";
571: this.textTimer.Start();
572: }
573: else if (e.GestureType == GestureType.WaveLeft)
574: {
575: this.GestureDetected = true;
576: this.GestureText = "Waved with left hand";
577: this.textTimer.Start();
578: }
579: else if (e.GestureType == GestureType.LeftSwipe)
580: {
581: this.GestureDetected = true;
582: this.GestureText = "Swiped left";
583: this.textTimer.Start();
584: }
585: else if (e.GestureType == GestureType.RightSwipe)
586: {
587: this.GestureDetected = true;
588: this.GestureText = "Swiped right";
589: this.textTimer.Start();
590: }
591: else if (e.GestureType == GestureType.Menu)
592: {
593: this.GestureDetected = true;
594: this.GestureText = "Menu";
595: this.textTimer.Start();
596: }
597: }
598:
599: /// <summary>
600: /// Defines the gestures.
601: /// </summary>
602: private void DefineGestures()
603: {
604: IRelativeGestureSegment[] waveRightSegments = new IRelativeGestureSegment[6];
605: WaveRightSegment1 waveRightSegment1 = new WaveRightSegment1();
606: WaveRightSegment2 waveRightSegment2 = new WaveRightSegment2();
607: waveRightSegments[0] = waveRightSegment1;
608: waveRightSegments[1] = waveRightSegment2;
609: waveRightSegments[2] = waveRightSegment1;
610: waveRightSegments[3] = waveRightSegment2;
611: waveRightSegments[4] = waveRightSegment1;
612: waveRightSegments[5] = waveRightSegment2;
613: this.gestures.AddGesture(GestureType.WaveRight, waveRightSegments);
614:
615: IRelativeGestureSegment[] waveLeftSegments = new IRelativeGestureSegment[6];
616: WaveLeftSegment1 waveLeftSegment1 = new WaveLeftSegment1();
617: WaveLeftSegment2 waveLeftSegment2 = new WaveLeftSegment2();
618: waveLeftSegments[0] = waveLeftSegment1;
619: waveLeftSegments[1] = waveLeftSegment2;
620: waveLeftSegments[2] = waveLeftSegment1;
621: waveLeftSegments[3] = waveLeftSegment2;
622: waveLeftSegments[4] = waveLeftSegment1;
623: waveLeftSegments[5] = waveLeftSegment2;
624: this.gestures.AddGesture(GestureType.WaveLeft, waveLeftSegments);
625:
626: IRelativeGestureSegment[] swipeleftSegments = new IRelativeGestureSegment[3];
627: swipeleftSegments[0] = new SwipeLeftSegment1();
628: swipeleftSegments[1] = new SwipeLeftSegment2();
629: swipeleftSegments[2] = new SwipeLeftSegment3();
630: this.gestures.AddGesture(GestureType.LeftSwipe, swipeleftSegments);
631:
632: IRelativeGestureSegment[] swiperightSegments = new IRelativeGestureSegment[3];
633: swiperightSegments[0] = new SwipeRightSegment1();
634: swiperightSegments[1] = new SwipeRightSegment2();
635: swiperightSegments[2] = new SwipeRightSegment3();
636: this.gestures.AddGesture(GestureType.RightSwipe, swiperightSegments);
637:
638: IRelativeGestureSegment[] menuSegments = new IRelativeGestureSegment[20];
639: MenuSegments1 menuSegment = new MenuSegments1();
640: for (int i = 0; i < 20; i++)
641: {
642: //gesture consists of the same thing 20 times
643: menuSegments[i] = menuSegment;
644: }
645:
646: this.gestures.AddGesture(GestureType.Menu, menuSegments);
647: }
648:
649: /// <summary>
650: /// Handles the Tick event of the ChangeTimer control.
651: /// </summary>
652: /// <param name="sender">The source of the event.</param>
653: /// <param name="e">The <see cref="System.EventArgs"/> instance containing the event data.</param>
654: void ChangeTimer_Tick(object sender, EventArgs e)
655: {
656: if (this.DeleteSkeleton != null)
657: {
658: this.DeleteSkeleton(this, null);
659: }
660: }
661:
662: /// <summary>
663: /// Resets the timer.
664: /// </summary>
665: public void ResetTimer()
666: {
667: this.ChangeTimer.Stop();
668: this.ChangeTimer.Start();
669: }
670:
671: /// <summary>
672: /// Notifies all change.
673: /// </summary>
674: private void NotifyAllChange()
675: {
676: this.NotifyPropertyChanged("Head");
677: this.NotifyPropertyChanged("LeftHand");
678: this.NotifyPropertyChanged("LeftHandX");
679: this.NotifyPropertyChanged("LeftHandY");
680: this.NotifyPropertyChanged("RightHand");
681: this.NotifyPropertyChanged("LeftWrist");
682: this.NotifyPropertyChanged("RightWrist");
683: this.NotifyPropertyChanged("LeftElbow");
684: this.NotifyPropertyChanged("RightElbow");
685: this.NotifyPropertyChanged("ShoulderCenter");
686: this.NotifyPropertyChanged("HipCenter");
687: this.NotifyPropertyChanged("LeftKnee");
688: this.NotifyPropertyChanged("RightKnee");
689: this.NotifyPropertyChanged("LeftAnkle");
690: this.NotifyPropertyChanged("RightAnkle");
691: this.NotifyPropertyChanged("LeftFoot");
692: this.NotifyPropertyChanged("RightFoot");
693: this.NotifyPropertyChanged("Spine");
694: }
695:
696: /// <summary>
697: /// Notifies the property changed.
698: /// </summary>
699: /// <param name="propertyName">Name of the property.</param>
700: private void NotifyPropertyChanged(string propertyName)
701: {
702: if (this.PropertyChanged != null)
703: {
704: this.PropertyChanged(this, new PropertyChangedEventArgs(propertyName));
705: }
706: }
707:
708: #endregion
709: }
710: }
Step 3 – Getting some sort of visual representation.
Before we do anything with the points that are coming through, the first thing to do is to simply display the camera feed from the Kinect. This is very easy to do.
In the XAML, add an image control (i.e. called “cameraFeed”). In the code, when the main window is created, set up an event handler for the Kinect’s ImageFrameReady property; and it will be called once the SDK has picked up an image.
Within the handler, create a ‘PlanarImage’ from the event args, and then set the image source of the image control to a new BitmapSource as seen below:
1: public MainWindow()
2: {
3: InitializeComponent();
4: //other stuff here
5: model.Kinect.ImageFrameReady += new EventHandler<Microsoft.Research.Kinect.Nui.ImageFrameReadyEventArgs>(kinect_ImageFrameReady);
6: }
7:
8: void kinect_ImageFrameReady(object sender, Microsoft.Research.Kinect.Nui.ImageFrameReadyEventArgs e)
9: {
10: PlanarImage image = e.ImageFrame.Image;
11: cameraFeed.Source = BitmapSource.Create(image.Width, image.Height, 96, 96, PixelFormats.Bgr32, null, image.Bits, image.Width * image.BytesPerPixel);
12: }
Now let’s do something with the actual points we have obtained.
What we wanted to do here was to create a very simple “image” of a skeleton on the screen.
One thing we did here was to disregard some points from the skeleton. We found it irrelevant to display a few points (i.e. shoulder blades, and sides of the hips etc.), as it didn’t really make the skeleton look or perform any differently, and by excluding them, improved the performance of the entire application.
So, with what remained, we used a combination of ellipses and lines to represent the skeleton.
Simple enough to start off with, add an items control to the page, and within its Data Template, place the 16 ellipses (representing each joint), and a line to connect to each set of relevant ellipses (sort of like the song…”the wrist bone is connected to the elbow”, and so forth).
Each ellipse was created as per below:
1: <Ellipse Height="14" Name="LeftHand" Stroke="Black" Fill="{Binding JointColor, Converter={StaticResource ColorToSolidColorBrushConverter}}" Width="14" Visibility="{Binding LeftHand, Converter={StaticResource JointToVisibilityConverter}}">
2: <Ellipse.RenderTransform>
3: <TranslateTransform X="{Binding LeftHand.Position.X, Converter={StaticResource KinectValueToScreenCoOrdinatesConverterX}, ConverterParameter=0}" Y="{Binding LeftHand.Position.Y, Converter={StaticResource KinectValueToScreenCoOrdinatesConverterY}, ConverterParameter=0}" />
4: </Ellipse.RenderTransform>
5: </Ellipse>
The lines were also very simple:
1: <Line StrokeThickness="3" Stroke="Black" StrokeEndLineCap="Round"
2: X1="{Binding LeftHand.Position.X, Converter={StaticResource KinectValueToScreenCoOrdinatesConverterX}, ConverterParameter=7}"
3: Y1="{Binding LeftHand.Position.Y, Converter={StaticResource KinectValueToScreenCoOrdinatesConverterY}, ConverterParameter=7}"
4: X2="{Binding LeftWrist.Position.X, Converter={StaticResource KinectValueToScreenCoOrdinatesConverterX}, ConverterParameter=7}"
5: Y2="{Binding LeftWrist.Position.Y, Converter={StaticResource KinectValueToScreenCoOrdinatesConverterY}, ConverterParameter=7}"
6: Fill="Black" />
By binding the ellipses to each relevant point in the Main View Model, and using the converter that we created earlier, you should obtain something similar to this:-
The Kinect sensor has picked up the skeletons of the two people standing in front of it. By then updating the collection of skeletons and updating the bindings of the items control source, the skeletons should be displayed correctly on the screen.
Now, as the frames come in, and the bindings update, the ellipses will seamlessly move around the screen as you move, and there you have it, you’re very own living skeleton!
In our next post, we will talk about how to write a gesture service with the SDK, which will recognize the gestures the skeletons on screen are making.
Written by Michael Tsikkos and James Glading