Partager via

Annotating Video with Ink using Silverlight

  • Article

Silverlight makes delivering video content in the browser very easy. Silverlight also has ink support built-in. So why not combine these two features to enable some great new scenarios?

I have put together a little sample using Silverlight v1.0 to record and play back ink annotations, synchronized with the video playback. Below is the key piece of XAML markup for this application. We are using an InkPresenter that hosts the MediaElement with the video and takes care of the ink renderering. Also note the empty storyboard here as this is the magic sauce for the synchronized ink playback.

<InkPresenter Name='inkPresenter' Canvas.Left='10' Canvas.Top='10'

MouseLeftButtonDown='onInkPresenterDown'

MouseMove='onInkPresenterMove'

MouseLeftButtonUp='onInkPresenterUp'>

<InkPresenter.Resources>

<Storyboard Duration="0:0:0" Completed="onStrokePlaybackTimerTick"

x:Name="strokePlaybackTimer" />

</InkPresenter.Resources>

<MediaElement Name='mediaElement' Source='bear.wmv'

Width='720' Height='480'

AutoPlay='False' MediaEnded='onMediaEnded'/>

</InkPresenter>

The relevant pieces in the JavaScript code behind the XAML (in file events.js), are the following functions:

  • ConvertInkToString(strokes) - a function to serialize the point data for the ink strokes into a string for persistence purposes
  • ConvertStringToInk(serializedStylusPoints) - the reverse function that returns a StrokeCollection for a serialized ink string
  • onStrokePlaybackTimerTick() - during playback this function is called on every frame; when it is time to play back a stroke, we do so point by point

The full source code is attached to this post. I have also uploaded the sample to the Silverlight Streaming service, so I can pull it directly into my blog here. To check it out, just click the button below to load the sample. Once the video is loaded, you can click the 'Playback' button to replay my pre-recorded ink annotation - or you can click 'Start Recording' to record your own annotations, and then play them back afterwards.

VideoAnnotation.zip

Comments

  • Anonymous
    November 15, 2007
    PingBack from http://www.gottabemobile.com/InkingOnVideoWithSilverlight.aspx

  • Anonymous
    November 15, 2007
    Microsoft&#39;s Stefan Wick continues to pump out some great Tablet PC ink and Silverlight stuff, this

  • Anonymous
    June 22, 2009
    This is a great example but it seems to be using a older version of silverlight.  I tried to make work using blend 2 or VS 2008 and I could not get it to run.   If not too much trouble can you update the sample code in the zip to either load in Blend 2 or 3 or VS 2008 or 2010