The modern.ie site also has a really cool way to test with various versions of IE, by using VM technology. In addition to versions of Internet Explorer from 6 through 11, you can get VM images for Mac or Linux based virtual machines. Check out virtualization tools, also on modern.ie. 

So with that, I'll say 'see ya'

-Jay

Recently, I've gone back and put those examples onto our sample server to eliminate the need to copy and paste code to see the API in action. I came across three samples that took a step just a little farther than just figuring out how the API was called. These examples actually showed some key issues, and really helped my own understanding. Additionally, the output from two of them were used as screen shots to illustrate the concepts.

lineCap property

The first example was the lineCap property which specifies how to terminate a line. You can assign one of three string values; butt, round, and square. The butt value tells canvas to not put any cap on the end of a line. The round value tells it to put on a round end, and square makes a square end. Round and square both extend the line approximately 1/2 the line width, for each cap.

That little bit of extra space could cause some problems, as the caps add to the specified line length. This is an issue if you're trying to accurately place your lines. I explained that concept with words in the topic, but realized a picture tells a better story. To do that, I created an example showing the concept. I added some vertical lines to show what the exact lengths of the lines, and how much the caps add.

arcTo method

The next example is the arcTo method.

The arcTo method draws an arc between two imaginary tangents, as the W3C spec describes it. This description required me to dig back into what I should have remembered from my high school geometry class (I didn't really pay attention).

For the example I drew two arcs. One made a nice rounded corner between two straight lines. The other didn’t complete the rounded corner because the tangents were closer together. I drew both the straight lines (in black), and the imaginary lines (in blue) to show the relationship. Maybe for you it would be obvious, but at the time it helped me understand how all the parameters worked together.

miterLimit property

Lastly, the miterLimit property. This property is a ratio of the angle when two lines come together. The ratio describes whether the desired effect of the lineJoin miter happens property. LineJoin specifies how canvas draws the intersection when two lines come together (say on a triangle). LineJoin’s options are bevel, round, and miter. When using a bevel or round lineJoin, the miterLimit setting has no effect.

However when you set lineJoin to miter (which brings the two lines to a point), miterLimit controls whether canvas draws a point or a squared edge corner. If the calculated ratio ((lineWidth / 2 ) / miter length) is below the miterLimit, you get a squared off edge. 

Experimenting with different hardcoded lineWidths and miterLimits didn't show the difference very well. Taking one of the examples, I substituted HTML5 slider controls so I could independently control the line width, angle of the two lines, and the miterLimit. If you're using IE9 then you'll see text boxes instead of sliders. Sorry, it's worth the upgrade to a later version of IE.

Viola! I was able to get the point to disappear and reappear, based on the values I slid into place.

Here’s the final example:

See the code

There are lots of samples that show how to use an API in the reference content. However these are among my favorites as they show how to write the syntax, and illustrate some of the concepts of the API.

For the code behind these examples, check out:

lineCap property

arcTo method

miterLimit property

Let me know what you think!

A few days later, Sylvia, one of the editors for the media spec brought it up again. I posted a copy of my example (and got smacked down for not including more formats) to show it worked in at least Chrome and IE. My example only had an MP4 format video, as I was linking to one on the IE Testdrive site. Anyway, long story short, Phillip reported back that test was fixed, and indeed both IE and Chrome passed the test.

What's the addTextTrack do?

Normally to get closed captioning, a video publisher would create a separate WebVTT or TTML format file (see Closed captioning, it’s not just for accessibility! TextTrack in HTML5 video for more on these formats). You can create different files for different languages, or purposes, such as scripting external content, etc.

But the problem is that you have to prepare the text track file ahead of time. Enter addTextTrack! As specified in the W3C, it lets you create textTracks as the content is playing. You can create the content programmatically based on input or other factors, or you could download updated content from your website using XML Header Request (XHR). You could even use addTextTrack as a work around for servers that don't know what to do with a WebVTT or TTML MIME type. One man's text is another man's captions, so to speak.

For an addTextTrack example, I create a simple caption track displaying the numbers 0 to 30 displayed every 5 seconds. It's very contrite, but shows the concept. See the example live online. Note that the online example only has an MP4 file link.

This source code has an MP4 and an OGG format video file. The source is a link to Archive.org and an old BBC technology series called Bad Influence. 

Legal note: Most of the content on the archive.org site is either creative commons license, or public domain. All rights of these clips remain with either archive.org or the BBC, and are here only for demonstration purposes.

This example works in Internet Explorer 11 and the latest version of Chrome. The video plays but the tracks don't show on Firefox. While addTextTrack is in the spec, the TextTrackCue constructor isn't. The addTextTrack constructor has been dropped in favor of the VTTCue constructor. I'm now looking for addTextTrack code that will work on other browsers. Suggestions?

<!DOCTYPE html >

<html >
  < head>
    <title>Add Text Tracks example</title>
< /head>
< body>

<video id="video1" controls="controls" muted="muted">
     <!-- change to your own mp4 video file -–> 
     <!-- show using multiple file formats -->

  <source src="https://archive.org/download/bad_influence_se4ep13/Bad.Influence.se4ep13.mp4" /> 
  <source src="https://archive.org/download/bad_influence_se4ep13/Bad.Influence.se4ep13.ogv" />

  HTML5 Video not supported
< /video>

<script>
    var video = document.getElementById("video1");
    var startTime, endTime, message;
    var newTextTrack = video.addTextTrack("captions", "sample");
    newTextTrack.mode = newTextTrack.SHOWING; // set track to display
    // create some cues and add them to the new track
    for(var i=0;i<30;i++){
        startTime =  i * 5 ;
        endTime = ( (i * 5) +  5);      
        message =  "This is number " + i;

        // this constructor deprecated according to the new spec      
        newTextTrack.addCue(new TextTrackCue(startTime, endTime, message));
    }
    video.play();
  </script>
< /body>
< /html>     

The 2D Context spec is in the final stages of going to Candidate Recommendation (CR), and has accessibility issues that are still being worked on. The canvas editors and the accessibility editors are getting together in a task force to talk about hit testing and focus rings. drawSystemFocusRing, which has been renamed  drawFocusAsNeeded. Focus rings is pretty much set.

Hit regions are still under discussion. They're a cool feature that if implemented, lets a developer define controls made of paths on the canvas, and associates them with controls (like buttons) that live in the fallback area between the <canvas> and </canvas> tags.

  Currently if a browser supports canvas, then what's between the tags is ignored, mostly. For example this snippet: 

<canvas id="mycanvas">

   Sorry, this browser or mode does not support canvas.

</canvas>

Obviously, if the browser doesn't support canvas, the fallback code (or text in this case), is displayed. You can also put interactive elements, like buttons in that space, like this:

<canvas id="mycanvas">

   <button id="mybutton">Click me</button>

</canvas>

Visually, if your browser supports canvas you're not going to see the button. If canvas isn't supported, then the button will show. However, there's a third option that only people who use assistive technology might know about. If your browser supports canvas, and it has say a screen reader hooked up, the button is "visible" to the reader.

Hit regions lets you put your button inside the canvas tags, create a button on the canvas (say using fillrect()), and link the two together as a hit region. The idea is that when the control gets focus, it can optionally alert the accessibility equipment. Additionally there's some updates (still discussing it) to mouse events so when you click on an area of the canvas that has a hit region, it sends a mouse message to the associated control that's between the canvas tags.

Now this will go a long way to making canvas accessible. But, if we can get the mouse events in, that's a win for everyone. You'll be able to easily create an interface by using elements in the fallback area, put some splashy graphics on the canvas, and have it work just like a regular UI. That would be cool.

We're still talking about hit regions, but hopefully we'll get the last things nailed down this week and move the spec back to CR. More later.

I hadn't tried that before, so I experimented with the focus and blur events. It turned out to be relatively easy, as long as I watched for events that called other events.

Using the focus and blur events

The focus and blur events aren't directly related to the media (audio or video) object, they're actually on the window object. The focus event fires when a webpage gets focus (that is, ready to interact with you). The blur event fires when you lose focus. Using them is pretty straight forward:

// when you leave a page

window.addEventListener("blur", function(){
   //do something here
}, false);

// when you return (or open) to a page

window.addEventListener("focus",function(){
  // do something else here
}

Now you need to add code to the "do something part.

To test the examples here open two tabs, one with the sample, and another page (anything will do).

Pause video on leaving, pickup when you come back

For simplicity, I'm using an audio element, but you can do this as well with video. This example plays a clip of audio and pauses when you go to another tab or even switch to another app. The "wasPlaying" flag tracks whether the audio was playing or not when you lost focus.

The one issue I had is how to keep the "wasPlaying" flag current. The best way is to use the "play" or "pause" events to set the flag. We've used this technique in previous media posts where we change the Play button text. In the "play" event handler, we set "wasPlaying" to true. In the "pause" event handler, we set it to false.

In addition to setting the "wasPlaying" flag, we need an additional flag to tell us that we are leaving when we call pause. I did this because when you call pause, it fires the pause event, which in turn sets  the "wasPlaying" flag to false. If we didn't track that we were leaving, we'd pause the audio when we lost focus. However, when we return, the "wasPlaying" flag is set to false in the "pause" event, and we wouldn't call play() to get started again.

Confused? Try commenting out the "leaving" flag and you'll see what happens. We also always initialize the leaving flag, because an uninitialized flag can be unpredictable.  Try this example online.

<!doctype html> 
< html>
< head>
    <title>Audio Element Sample</title>
< /head>
< body>
    <h1>Audio Element Sample</h1>
    <!-- Display the audio player with control buttons. -->
    <audio id="myaudio" src="https://testdriveie9.wise.glbdns.microsoft.com/ietestdrivecontent/Musopen.Com Symphony No. 5 in C Minor, Op. 67 - I. Allegro con brio.mp3" controls autoplay loop>
        HTML5 audio not supported
    </audio>
    <script>
      var audio =document.getElementById("myaudio");

      var wasPlaying; // set this to true sometime if playing
      var leaving = false;

      window.addEventListener("blur", function () {
        leaving = true;
        if (wasPlaying) {
          audio.pause();
        }
      }, false);

      window.addEventListener("focus", function () {
        if (wasPlaying == true) {
          audio.play();
        }
        leaving = false;
      }, false);

      audio.addEventListener("play", function () {
        wasPlaying = true;
      }, false);

      audio.addEventListener("pause", function () {
        if (!leaving) {
          wasPlaying = false;
         }
      }, false);

    </script>
< /body>
< /html>

Cut the volume when you leave

Sometimes you don't want to stop the music when you leave a page, just turn it down a little. This example is actually simpler than pausing the music, because we don't need to worry about setting a special flag when we fire pause/play events.

To adjust the volume, we set the volume property on the media object. Volume is set between 0 and 1. When we lose focus, we're setting the volume to 1/3 what it was. You can set it to anything (even full volume to get users back to your page if you want). We stash the current volume in a global variable when we lose focus so we can reset it when we come back.

Try this example online.

<!doctype html> 
< html>
< head>
    <title>Audio Element Sample</title>
< /head>
< body>
    <h1>Audio Element Sample</h1>
    <!-- Display the audio player with control buttons. -->
    <audio id="myaudio" src="https://testdriveie9.wise.glbdns.microsoft.com/ietestdrivecontent/Musopen.Com Symphony No. 5 in C Minor, Op. 67 - I. Allegro con brio.mp3" controls autoplay loop>
        HTML5 audio not supported
    </audio>
    <script>

      var wasPlaying = false; // set this to true sometime if playing
      var wasVolume;   // save the current volume when we lose focus
       var audio = document.getElementById("myaudio");

      window.addEventListener("blur", function () {
        if (wasPlaying) {
          wasVolume = audio.volume;   // save the current volume
          audio.volume = wasVolume / 3; // or any volume setting
        }
      }, false);

      window.addEventListener("focus", function () {
         // if we were playing when we lost focus, reset volume when we come back
        if (wasPlaying == true) { 
          audio.volume = wasVolume;
        }
       }, false);

      audio.addEventListener("play", function () {
        wasPlaying = true;
      }, false);

      audio.addEventListener("pause", function () {
          wasPlaying = false;
      }, false);

    </script>
< /body>
< /html>

Some recent feedback wasn't bad, but rather a suggestion. "Luke" posted a comment on my topic "Using JavaScript to Control the Audio Object". He said that he'd like to change songs on the fly with a pick list. The original example had a text box that you could write in your own audio file to play, so it really didn't fit the bill.

On the Internet Explorer Testdrive site, there's a slick multi-file player that uses SVG, audio, and XML. For something a little simpler (seems to be a theme song in here somewhere), the mods I did to my original single song player might do the trick.

Specifying a source file on an audio element

There are two ways to assign a URL to an audio element in HTML, use the src attribute on the audio element or use the source element between the audio tags. The src attribute is good when you have just a single file that you're going to play.

Here's an ultra simple audio player:

<!DOCTYPE html>
< html>
  <head>
    <title>Simple Audio Example</title>
< /head>
< body>
  <audio control src="songfile.mp3" >Not supported</audio>
< /body>
< /html>

If you want to give the audio element a few choices of format, such as mp3, aac, or ogg, then you'll want to use the source elements.

<!doctype html>
< head>
    <title>
         Multiple format audio example
    </title>
    <!-- Uncomment the following meta tag if you have issues rendering this page on an intranet site. -->   
    <!--  <meta http-equiv="X-UA-Compatible" content="IE=9"/> -->
< /head>
< body>
    <h1>
         Using source to support multiple audio formats
    </h1>
    <!-- The browser will automatically choose the format it supports. -->
    <audio controls="true">
        <source src="demo.mp3" type="audio/mp3">
        <source src="demo.ogg" type="audio/ogg">
        <source src="demo.aac" type="audio/mp4">
        <!-- If no support at all. -->
        HTML5 audio not supported
    </audio>
< /body>
< /html>

The source element takes a single src attribute, but you can stack up as many elements as you have formats for a given file. The <audio> element picks the first format it can play. If you try to put three different mp3 songs in <source> elements, only the first will play, every time. Unlike the track element which similarly resides between the video element tags, you have no control, selection is automatic.

Swapping different audio source files

To set up multiple sources, like a playlist, you need to write a little JavaScript. When you want to change songs on an audio control, you can simply assign a new URL to the src property and call play(). There is one gotcha that can pop up. If you originally specified a song with the source element, you need to add one step, call load() before calling play().

<!DOCTYPE html>

<html>
< head>
    <title>Simple song switcher</title>
< /head>
< body>
  <audio controls="controls" id="myaudio">
    source src="firstaudiofile.mp3" type="audio/mp3">
    Sorry, HTML5 audio isn't supported.
  </audio><br />
  <button id="mybutton">Change songs</button>
  <script>
    // Handle the button click and play new file
    document.getElementById("mybutton").addEventListener("click", function () {
      // Get audio object from audio element
      var oAudio = document.getElementById('myaudio');
      // double check support, and set the
      if (oAudio) {
        oAudio.src = "nextaudioURL.mp4";
        oAudio.load();
        oAudio.play();
      }
    }, false);
  </script>
< /body>
< /html>

That's pretty much all you need to do. Where code could get complex is in the song switching algorithm.

A little more of a demo

Here's a dressed up version of the original sample from Using JavaScript to control the audio object.

Not much to look at, but you can choose between 5 songs, or load your own. The code is below.

The example has a simple UI that consists of an input field, a select box, and a few buttons. The songs are hard coded into the select box, with the text and URL for a song as a data pair on each option element.

All events are handled by addEventListeners in the JavaScript code. The play button calls play or pause, depending on its current state. The load method is called first if play is being called.

To switch the Play button label between "Play" and "Pause", we've added event listeners to the pause and play events, rather the play button code itself. This allows the button text to follow the actual state of the audio control. While the initial code doesn't expose the audio controls, you could add the intrinsic controls by adding the controls="controls" attribute to the audio element. The play button that switches labels would still stay in sync.

<!DOCTYPE html>
< html>
 
  <head>
    <title>HTML5 audio playlist demo </title>   
    <!-- Uncomment the following meta tag if you have issues rendering this page on an intranet site. -->   
    <!--  <meta http-equiv="X-UA-Compatible" content="IE=edge"/> --> 
  </head>
 
  <body>
    <h1>Audio playlist demo</h1>
    <p><label>Enter an audio file URL:
      <input type="text" id="audiofile" size="80" />
      </label>
    </p>
    <p>Or pick from this list:</p>
    <select id="myselect">
      <option value="https://samples.msdn.microsoft.com/Workshop/samples/media/Musopen_Com_Symphony_No_5_in_C_Minor_Op_67_-_I_Allegro_con_brio.mp3">Symphony No. 5 in C Minor, Op. 67 - I. Allegro con brio (Ludwig van Beethoven)</option>
      <option value="https://samples.msdn.microsoft.com/Workshop/samples/media/Musopen_Com_Symphony_No_5_in_C_Minor_Op_67_-_II_Andante_con_moto.mp3">Symphony No. 5 in C Minor, Op. 67 - II. Andante con moto (Ludwig van Beethoven)</option>
      <option value="https://samples.msdn.microsoft.com/Workshop/samples/media/Musopen_Com_Symphony_No_5_in_C_Minor_Op_67_-_III_Allegro.mp3">Symphony No. 5 in C Minor, Op. 67 - III. Allegro (Ludwig van Beethoven)</option>
       <option value="https://samples.msdn.microsoft.com/Workshop/samples/media/Musopen_Com_Symphony_No_5_in_C_Minor_Op_67_-_IV_Allegro.mp3">Symphony No. 5 in C Minor, Op. 67 - IV. Allegro (Ludwig van Beethoven)</option>
      <option value="https://samples.msdn.microsoft.com/Workshop/samples/media/Musopen_Com_Piano_Concerto_in_A_Minor_Op_16_-_I_Allegro_molto_moderato.mp3">Piano Concerto in A Minor, Op. 16 - I. Allegro molto moderato (Edvard Grieg)</option>
    </select>
    <p>All selections performed by Skidmore College Orchestra, available on https://musopen.org/.</p>
    
    <audio id="myaudio">
      HTML5 audio not supported
    </audio>

    <button id="play">
      Play
    </button>
   
    <button id="rewind">
      Rewind
    </button>
   
    <button id="forward">
      Fast forward
    </button>
   
    <button id="restart">
      Restart
    </button>

    <script>
    // Global variable to track current file name.
      var currentFile = "";
      var oAudio = document.getElementById('myaudio');
      var btn = document.getElementById('play');
      var audioURL = document.getElementById('audiofile');
      var mySelect = document.getElementById("myselect");

      document.getElementById("play").addEventListener("click", function () {
        // Check for audio element support.
        if (oAudio) {
          try {
             // Tests the paused attribute and set state.
            if (oAudio.paused) {
              //Skip loading if current file hasn't changed.
              if (audioURL.value !== "") {
                 if (audioURL.value !== currentFile) {
                   oAudio.src = audioURL.value;
                  currentFile = audioURL.value;
                  oAudio.load(); // load the file
                }
              } else {
                var index = mySelect.selectedIndex;
                var playThis = mySelect.options[index].value;
                if (playThis !== currentFile) {
                  oAudio.src = playThis;
                   currentFile = playThis;
                  oAudio.load(); // load the file
                }
              }

              oAudio.play();
            }
            else {
               oAudio.pause();
            }
          }
          catch (e) {
            // Fail silently but show in F12 developer tools console
            if (window.console && console.error("Error:" + e));
          }
        }
      }, false);

      // These two event listeners handle changing the play button from Pause to Play and back.
      oAudio.addEventListener("pause", function () { btn.textContent = "Play"; }, false);
      oAudio.addEventListener("play", function () { btn.textContent = "Pause"; }, false);

      // Rewinds the audio file by 30 seconds.
      document.getElementById("rewind").addEventListener("click", function () {
        if (oAudio) {        // Check for audio element support.
          try {
            oAudio.currentTime -= 30.0;
           }
          catch (e) {
            // Fail silently but show in F12 developer tools console
            if (window.console && console.error("Error:" + e));
          }
        }
      }, false);

      // Fast forwards the audio file by 30 seconds.

      document.getElementById("forward").addEventListener("click", function () {
        if (oAudio) {        // Check for audio element support.

          try {
            oAudio.currentTime += 30.0;
          }
          catch (e) {
            // Fail silently but show in F12 developer tools console
            if (window.console && console.error("Error:" + e));
          }
         }
      }, false);

      // Restart the audio file to the beginning.
      document.getElementById("restart").addEventListener("click", function () {
        if (oAudio) {        // Check for audio element support.
          try {
            oAudio.currentTime = 0;
          }
          catch (e) {
            // Fail silently but show in F12 developer tools console
            if (window.console && console.error("Error:" + e));
          }
        }
      }, false);

      // On changing songs, pause video and call Play function again
      mySelect.addEventListener("change", function () {
        oAudio.pause();
        btn.click();
      }, false);
    </script>
  </body>
< /html>

The video at the top is the original, and the canvas with the fill pattern is the big block below. The small video pattern is repeated in both the vertical and horizontal direction. If you run the video, you'll see that it all moves simultaneously. You can substitute an video file of your own, since this one isn't that action packed. Note: You need to use a hosted .MP4 file, not a random file from YouTube. For the most part, the URL you get in YouTube is a link to a page with an embedded video, not the video itself.

You can set up fills to repeat vertically:

or horizontally:

Or not at all:

Picking how to repeat the pattern is up to you.

If you're familiar with using fills in Canvas, you're only a few steps away from filling with a video. A video fill is essentially a series of still frames swapped out as fast as the computer can do it. The result is a fill with full motion video. Here's how it works.

Fill'r up, getting a shape filled

For a quick refresher, this example shows how to fill a rectangle on a canvas.

<!DOCTYPE html>
<html>
< head>
  <title>Simple rectangle fill</title> 
< /head>
< body>
  < canvas width="1080" height="800" id="myCanvas">
    Sorry, canvas isn't supported in this browser or mode
  </canvas>
  <script>
    var canvas = document.getElementById("myCanvas");
    if(canvas.getContext){
      var ctx = canvas.getContext("2d");     
      ctx.fillStyle='red';
      ctx.fillRect(100, 100, 900, 600);
      }
  </script>
< /body>
< /html>

Picture a fill

Now to do the same thing with an image. This code example adds a pattern object using an image and the repetition direction. The original <img> element is hidden through a CSS style (display: none;).

<!DOCTYPE html>
<html>
< head>
  <title>Simple rectangle pattern fill</title> 
  <style>
    img {
    display:none;
    }
  </style>
< /head>
< body>
  <canvas width="1080" height="800" id="myCanvas">
    Sorry, canvas isn't supported in this browser or mode
  </canvas>
  < img id="pix" src="https://go.microsoft.com/fwlink/?LinkID=199028" />
  <script>
    var canvas = document.getElementById("myCanvas");
    if (canvas.getContext) {
      var image = document.getElementById("pix");
       image.addEventListener("load", function () {
        var ctx = canvas.getContext("2d");
        var pattern = ctx.createPattern(image, "repeat");
        ctx.fillStyle = pattern;
         ctx.fillRect(100, 100, 900, 600);
      }, false);
       }
  </script>
< /body>
< /html>

We added a couple of statements, the pattern object variable, an <img> tag with a NASA image, and a little CSS. The CSS simply turns off the display for the <img> tag. In the JavaScript code, we get the image object of the hidden image. Using createPattern, we create a pattern with a repetition of both horizontal and vertical.

We also captured the load event for the image because the image is coming across time and space of the Internet, and might not be fully loaded into the <img> tag when we need it. Once load fires, we then get the canvas context, and do the fill.

Video, one frame at a time

To create a video pattern, we swap out the <img> tag with a <video> tag. Using the same code as the image pattern but with the video element. It works, but only we just get a single video frame each time fill is called. In this example, try clicking the Show video frame button as the video changes.

<!DOCTYPE html>
< html>
< head>
  <title>Simple rectangle video frame fill</title> 
  <style>
     video {
    /*display:none;*/
    }
  </style>
< /head>
< body>
 
  <video id="myVideo" autoplay ="autoplay" controls="controls" width="200" height="160" src="https://ie.microsoft.com/testdrive/ieblog/2011/nov/pp4_blog_demo.mp4">
    Sorry, video isn't supported in this browser or mode
  </video>                                      
  <button id="doVideo">Do video capture</button><br />
  <canvas width="1080" height="800" id="myCanvas">
    Sorry, canvas isn't supported in this browser or mode
  </canvas>
 
  <script>
    document.getElementById("doVideo").addEventListener("click", fillVideo, false);

    fillVideo();

    function fillVideo() {
      var canvas = document.getElementById("myCanvas");
      if (canvas.getContext) {
        var video = document.getElementById("myVideo");
        var ctx = canvas.getContext("2d");
        var pattern = ctx.createPattern(video, "repeat");
        ctx.fillStyle = pattern;
        ctx.fillRect(100, 100, 900, 600);
      }
    }
  </script>
< /body>
< /html>

Not very dynamic, as it only grabs a single video frame.

Motion potion

To create the magic of full motion video, we'll add an animation loop. We'll also add a button to start and stop the video, since the original video element is hidden in this demo. This example puts it all together.

<!DOCTYPE html>
< html>
< head>
  <title>Simple rectangle video frame fill</title> 
  <style>
    video {
      /* Turn off the original video element*/
    display:none;
    }
  </style>
< /head>
< body>
  <!-- Set up and play video -->
  <video id="myVideo" autoplay ="autoplay" src="https://ie.microsoft.com/testdrive/ieblog/2011/nov/pp4_blog_demo.mp4">
    Sorry, video isn't supported in this browser or mode
  </video>                                      
  <button id="stop">Stop video</button><br />
  <canvas width="1280" height="800" id="myCanvas">
    Sorry, canvas isn't supported in this browser or mode
  </canvas>
 
  <script>
    var canvas = document.getElementById("myCanvas");
    if (canvas.getContext) {
      // Canvas setup
      var ctx = canvas.getContext("2d");
      var video = document.getElementById("myVideo");
      var timerID;
       var w = canvas.width;
      var h = canvas.height;

      function animateVideo() {

        // Stop loop when video ends
         if (video.paused || video.ended) {
          return false;
         }
        // Create a repeat video pattern
        pattern = ctx.createPattern(video, "repeat"); // Get the video frame  
        ctx.fillStyle = pattern;  // Assign pattern as a fill style.
        ctx.fillRect(0, 0, w, h); // Fill the canvas.

        // Restart the animation
        timerID = window.requestAnimationFrame(animateVideo);
      }
      var button = document.getElementById("stop");
      // Start or stop video based on video state
      button.addEventListener("click", function () {
        if (video.paused) {
          video.play();
        } else {
          video.pause();
        }
      }, false);

      // Change button text based on the video state
      video.addEventListener("pause", function () {
        button.innerHTML = "Play";
      }, false);

      // Change button text based on the video state, and start animation
      video.addEventListener("play", function () {
        button.innerHTML = "Pause";
        animateVideo();
      }, false);
    }
  </script>
< /body>
< /html>

We've added a couple of elements for ease of use. The button handler for video control has two independent parts. In the first part, the "click" event listener watches for a button press, and then checks to see if the video is paused or not (the only two states we're interested in). If it's paused, it tells the video to play. If not (else), it tells the video to pause.

The second part has two event listeners that watch for the pause and the play events. If the pause event fires, it changes the button text to "Play." If the play event fires, it changes the button text to "Pause." We could have put the button changing code into the click event for the button, but run the risk of getting out of sync with the video. When the video starts (with autoplay), the button might not have the correct label. When we capture the play and pause events, the button text change is independent of the video controls,. So whether we're pausing with the button or the video element (if displayed), it's still in sync.

The secret sauce

For full motion video display, we need to refresh the fill pattern more than just pressing a button like we did in the last example. This is accomplished using the requestAnimationFrame method.

The requestAnimationFrame method is the recommended way of getting accurate animation timing. Previously JavaScript coders used the setTimeout method, which uses a fixed amount of time between frames. This works, but there are two problems. First, the UI might not be ready to render, so it skips frames (choppy animation). Second, when you leave the page, the animation is still running which eats up resources.

The requestAnimationFrame method solves both these problems. RequestAnimationFrame asks the system to let us know when it's ok to draw an image. It never tries to do something when it can't, so there are no skipped frames. When you leave the page, the animation is suspended until you return to the page, so no resources are used.

In our example, we start the animation loop from within the play event. This ensures that the animation loop starts when the video is playing. If we put the call to animateVideo outside the loop, it might try to display the video before it's fully started, in which case it'll exit and never run again.

Let's Play!

One cool thing with patterns, video or otherwise, is that you don't need to have a rectangle for your patterns. This example is a slight modification of the full motion video example. Rather than projecting into a rectangle, we're using a shape from an earlier blog post.

<!DOCTYPE html>
< html>
< head>
  <title>Simple rectangle video frame fill</title> 
  <style>
    video {
      /* Turn off the original video element*/
    display:none;
    }
  </style>
< /head>
< body>
  <!-- Set up and play video -->
  <video id="myVideo" autoplay ="autoplay" src="https://ie.microsoft.com/testdrive/ieblog/2011/nov/pp4_blog_demo.mp4">
    Sorry, video isn't supported in this browser or mode
  </video>                                      
  <button id="stop">Play</button>    <button id="change">Change fill rule</button> <br />
  <canvas width="1280" height="800" id="myCanvas">
    Sorry, canvas isn't supported in this browser or mode
  </canvas>
 
  <script>

    var canvas = document.getElementById("myCanvas");
    if (canvas.getContext) {
      // Canvas setup
      var ctx = canvas.getContext("2d");
      var video = document.getElementById("myVideo");
      var timerID;
      var w = canvas.width;
      var h = canvas.height;

      function animateVideo() {

        // Stop loop when video ends
        if (video.paused || video.ended) {
          return false;
        }
        // Create a repeat video pattern
        pattern = ctx.createPattern(video, "repeat"); // Get the video frame  
        ctx.clearRect(0, 0, canvas.width, canvas.height); // Clear area for repeated use
         ctx.beginPath();
        ctx.strokeStyle = 'blue';
         ctx.lineWidth = "5"
        ctx.moveTo(200, 110);
        ctx.lineTo(50, 110);
        ctx.lineTo(300, 310);
        ctx.lineTo(200, 10);
        ctx.lineTo(100, 310);
        ctx.lineTo(350, 110);
        ctx.lineTo(200, 110);
        ctx.stroke();
        ctx.fillStyle = pattern;  // Assign pattern as a fill style.        
        ctx.fill();
        // Restart the animation
        timerID = window.requestAnimationFrame(animateVideo);
      }

      var change = document.getElementById("change");
        change.addEventListener("click", function () {
         if (ctx.msFillRule == "nonzero") {
          ctx.msFillRule = "evenodd";
          change.innerHTML = "Change to non-zero";
        } else {
          ctx.msFillRule = "nonzero";
           change.innerHTML = "Change to even-odd";
        }
      },false);

      var button = document.getElementById("stop");
       // Start or stop video based on video state
      button.addEventListener("click", function () {
        if (video.paused) {
          video.play();
        } else {
          video.pause();
        }
      }, false);

      // Change button text based on the video state
      video.addEventListener("pause", function () {
        button.innerHTML = "Play";
      }, false);

      // Change button text based on the video state, and start animation
      video.addEventListener("play", function () {
         button.innerHTML = "Pause";
        animateVideo();
       }, false);
    }
  </script>
< /body>
< /html>

Fun, right? Now with a little imagination, you can fill all kinds of things with video, circles, triangles, or even text.

Give it a try.

-Jay

The full screen API is a collection of methods, properties, events and CSS elements that your app can use to control elements going full screen. It's a W3C and WhatWG standard. However the W3C spec is in working draft, and has been there since 2012, and the WhatWG spec, while it has a current date, pretty much looks the same. We might have to stick with the prefixes for a while.  

This table shows the fullscreen API available in Internet Explorer 11.

More info on these are available at Fullscreen API on MSDN.

Here’s a simple example that has three <divs>, an image, and a video element that you can click and make go full screen. It’s set up for Internet Explorer 11, Google Chrome, Mozilla Firefox, and Opera. Click any element to make it go full screen. Click the element a second time to exit full screen.

The example uses HTML classes to tag the elements that we want to make full screen. A simple loop runs on the collection of elements and creates click events which calls the makeFullFrame() function. The makeFullFrame function sorts out which prefixed version will work, and either requests full frame, or exits full frame.

All browsers have a way for the user to cancel full screen, as that’s one of the security items that the W3C spec calls out. However, I wanted a way to Initially I created separate full screen and exit full screen functions. Clicking the element made it full screen, and a button was supposed to exit full frame. The button used z-order to keep it on the top, but it turned out that didn't work because when you go to full frame, z-order is ignored.

While testing the makeFullFrame function to do double duty, I found I had to cancel bubbling on the click event. This was because anytime I clicked anything other than the first <div>, the click event would keep firing through all the child elements. Since all the elements are nested, that wouldn't work. The solution was to use stopPropagation() on the event on the first pass through.

The makeFullFrame function, other than testing for all the prefixed methods, also checks with the document object to see if an element is full screen. When any element is full screen in a page, the document.fullScreenElement attribute is set. Each browser has their own version of it, so we had to use prefixes there too. If the fullScreenElement is set, we then call the document.cancelFullScreen method. The W3C spec currently specifies exitFullscreen(), (IE uses msExitFullScreen), but the other browsers use a prefixed version of cancelFullscreen.

While each browser uses similar APIs, they don’t all behave the same. The W3C and WhatWG specs don't specify what elements expand to full screen, and which ones just block out the background.

This chart is the result of a quick test of the four browsers. Full screen means the element (img, div, or video) fills the screen completely. Same size means the element (div, img), stayed the same size as a normal view, but the background is blocked out. 

Browser	<div>	<img>	<video>	click to exit
Internet Explorer	Same size	Full screen	Full screen	Yes
Chrome	Same size	Same size	Full screen	Yes
Firefox	Full screen	Full screen	Full screen	Yes but you must accept the Allow option.
Opera	Same size	Same size	Full screen	Yes except on video

What’s interesting is that only Firefox expands a div to display full screen. All other browsers keep div elements the same size. One way around this (In Internet Explorer) is to use CSS and one of the new pseudo classes to fill the screen with the div (if that’s what you want).

Here’s how:

<style>

#test:-ms-fullscreen {
  position:absolute;
  left:0px;  
  top:0px;
  width: 100%;
  height: 100%;
}

</style>

Pretty slick, right? Another full screen pseudo element lets you either set a color or an image for the background for elements that don’t display edge to edge. Here’s how you call that one:

<style>

/* creates a background color of magenta */
#test:-ms-fullscreen::-ms-backdrop { 
   background-color: magenta;
}

</style>

<style>

/* creates a background image */
#test:-ms-fullscreen::-ms-backdrop { 
   background-image: url("image.jpg");    
}

</style>

When you view the example in Internet Explorer, you'll see a magenta background around the <divs>. You can modify the code to use a background-image as well. Unfortunately, not all browsers have jumped on the bandwagon for these two CSS pseudo goodies. 

So that's the fullscreen API in a nutshell. Take a look at the reference pages (linked from the table above, and a conceptual piece we did for IE11 launch. Note that the example is going under a facelift, since there were a bug or two in it. The following example is the "new and improved" version, and mirrors the online example.

<!DOCTYPE html>
< html >
< head>
    <title>Fullscreen API test</title>
    <style>
      video {
        /* Push video element over */
        position:relative;
        left: 350px;
      }
      #div1 {
        background-color:green;
        width:800px;
         height:500px;
      }
      #div2 {
        background-color:yellow;
        width:700px;
        height:400px;
      }
      #div3 {
        background-color:red;
        width:300px;
        height:300px;
      }
     
      div:-ms-fullscreen::-ms-backdrop {
        background-color:magenta;
      }
     </style>
 
< /head>
< body >
  <div id="div1" class="fullScreen" > Div1               
        <div id="div2" class="fullScreen" > Div2       
           <video class="fullScreen" controls="controls" loop="loop" autoplay="autoplay" width="300">
              <source src="demo.mp4" type="video/mp4" /> <!-- support for IE, Firefox, or Chrome -->
             <source src="demo.ogv" type="video/ogg" />
< !--             <source src="demo.webm" type="video/webm" />  -->  <!-- sample server doesn't like this -->
             Sorry, video isn't supported
           < /video>         
           <div id="div3" class="fullScreen" > Div3
             < img class="fullScreen" src="image.jpg"  />
           </div>
        </div>    
    </div>
   <script>

   // create click events for all the elements based on class.
   var fsClass = document.getElementsByClassName("fullScreen");
   for (var i = 0; i < fsClass.length; i++) {
     fsClass[i].addEventListener("click", function (evt) { makeFullScreen(evt); }, false);     
   }
   
   function makeFullScreen(evt) {
      // Test for each of the supported versions of full screen APIs and call
     // either requestFullscreen or cancelFullScreen (or exitFullScreen)
     //  Structure:
     //  Does the incoming target support requestFullscreen (or prefaced version)
     //  if (there is a fullscreen element)
     //      then cancel or exit
     //  else request full screen mode

     var divObj = evt.target;  //  get the target element
    
     if (divObj.requestFullscreen)  
       if (document.fullScreenElement) {
         document.exitFullscreen();      
       } else {
         divObj.requestFullscreen();
       }
     else if (divObj.msRequestFullscreen)
       if (document.msFullscreenElement) {
         document.msExitFullscreen();
       } else {
        divObj.msRequestFullscreen();
       }
     else if (divObj.mozRequestFullScreen)
       if (document.mozFullScreenElement) {
         document.mozCancelFullScreen();
      } else {
        divObj.mozRequestFullScreen();
     }
     else if (divObj.webkitRequestFullscreen)
       if (document.webkitFullscreenElement) {
         document.webkitCancelFullScreen();
        } else {
         divObj.webkitRequestFullscreen();
       }
      //  stop bubbling so we don't get bounce back
      evt.stopPropagation();

    }

  </script>
< /body>
< /html>

Closed captioning was originally conceived for hard of hearing viewers and offering programming in other languages without re-dubbing. Captioning can be a word for word transcription or translation of screen content, or it can be used for additional content like titles, locations, or commentary.

The track element

To add closed captioning, add a track element and point to a track file in your HTML5 video code. This example simply plays a video with a track file.

<!DOCTYPE html>
<html>
  <head>
  <title>Simple track example</title>
  </head>
  <body>
    <video id="video1" controls autoplay>
      <source src="https://ie.microsoft.com/testdrive/Videos/BehindIE9ModernWebStandards/Video.mp4"  >
      <track id='track1' label='English captions' src="entrack.vtt" kind='subtitles' srclang='en' default >   
    </video>
   
  </body>
</html>

Pretty cool, huh? Not a lot of extra code, just add the track element and you’ve got closed captioning.

The built in controls that appear when you use the controls attribute on the video element lets you turn closed captioning on or off. If you’ve got more than one track you can select between them as well.

 Here’s an example of adding three translation tracks:

<video id="video1" controls autoplay loop>
  < source src="video.mp4" type="video/mp4">
  <track id="enTrack" src="entrack.vtt" label="English" kind="subtitles" srclang="en" default>
  <track id="esTrack" src="estrack.vtt" label="Spanish" kind="subtitles" srclang="es">
  <track id="deTrack" src="detrack.vtt" label="German" kind="subtitles" srclang="de">
  HTML5 video not supported
</video>

Inside the video element it’s got a source element that points to the video file, and three track elements that point to track files. These tracks are written in WebVTT, a format for closed captioning. Lastly a message to display if HTML5 video isn’t supported.

Internet Explorer supports two formats for closed captioning, WebVTT and TTML.

WebVTT is a reasonably simple format that provides the in and out timing points to insert text, and the text.

WEBVTT

00:00:01.878 --> 00:00:05.334
Good day everyone, my name is John Smith

00:00:08.608 --> 00:00:15.296
This video will teach you how to
build a sand castle on any beach

WebVTT has a few more features as outlined in the W3C community group spec, but not all are supported in IE.

The other format, TTML is an XML based format. A simple TTML file looks like this:

<?xml version='1.0' encoding='UTF-8'?>
<tt xmlns='https://www.w3.org/ns/ttml' xml:lang='en'>
<body>
<div>

<p begin="00:00:01.878" end="00:00:05.334" >Good day everyone, my name is John Smith</p>
<p begin="00:00:08.608" end="00:00:15.296" >This video will teach you how to<br/>build a sand castle on any beach</p>
</div>

</body>
</tt>

This is the original code that was supported in Internet Explorer 10. In IE11, support for Simple Delivery Profile (SDP) caption styling was added. SDP gives some pretty cool abilities to place the text within the video frame, change text color, and background, and control stroke color and width. The full spec is here:

TTML Simple Delivery Profile for Closed Captions (US)

The TTML gets a little more complicated, but it can really add power to your captioning.

<?xml version="1.0" encoding="utf-8"?>
<tt xml:lang="en-us" xmlns="https://www.w3.org/ns/ttml"
    xmlns:s="https://www.w3.org/ns/ttml#styling"
    xmlns:p="https://www.w3.org/ns/ttml#parameter">
  <head>
     <p:profile use="https://www.w3.org/ns/ttml/profile/sdp-us"/>
     <styling>
      <!-- define styles for text color and position -->
      <style xml:id="bottomMidStyle" s:textAlign="center" s:textOutline="red 1px" s:backgroundColor="#ff000044"
       s:color="#ffffffff" s:origin='20% 78%' s:extent='30% 10%'/>
      <style xml:id="topMidStyle" s:textAlign="center" s:textOutline="black 1px" s:backgroundColor="#00ff0088"
       s:color="#ff11ffff" s:origin='20% 40%' s:extent='60% 18%'/>
      <style xml:id="topLeftStyle" s:textAlign="left" s:textOutline="blue 1px" s:backgroundColor="transparent"
       s:color="#ff11ffff" s:origin='10% 10%' s:extent='30% 10%'/>
      <style xml:id="bottomRightStyle" s:textAlign="right" s:textOutline="black 1px" s:backgroundColor="white"
       s:color="green" s:origin='70% 70%' s:extent='30% 10%'/>
    </styling>

    <layout>
      <!-- define regions for locating text -->
      <region xml:id="bottomMid" style='bottomMidStyle' />
      <region xml:id="topMid" style='topMidStyle' />
      <region xml:id="topLeft" style='topLeftStyle' />
      <region xml:id="bottomRight" style='bottomRightStyle' />
    </layout>
  </head>
  <body>
    <div style='defaultFont'>
      <p region="bottomMid" begin='00:00:00.101' end='00:00:05.000'> This is a Pop-up caption</p>
      <p region="topMid" begin='00:00:05.000' end='00:00:10.000'> This is another Pop-up caption</p>
      <p region="topLeft" begin='00:00:10.000' end='00:00:15.000'> Hello from up top</p>
      <p region="bottomRight" begin='00:00:15.000' end='00:00:20.000'> And back down</p>
   </div>
  </body>
</tt>

Using SDP is no different than plain captioning:

<!DOCTYPE html>
<html>
    <head>
        <title>SDP Test</title>      
    </head>
    <body>
        <video src="video.mp4" controls muted autoplay width="800">
            <track src="SDPTest.ttml" label="SDP Examples" default/>
        </video>
    </body>
</html>

That’s it.

Adding code for supporting a track file is not very hard. However, you do need to work a little to create the .vtt or .ttml file. To make that a little easier, check out HTML5 Video Caption Maker.  How to use Caption Maker and more on Text Track is described in Make your videos accessible with Timed Text Tracks.

Caption Maker lets you load a video file and step through it, typing or pasting in captions or comments. Try it first with the Load Sample button. This loads up an MP4 file to play with. Once the video is loaded, you get the option to load a sample VTT file. You can load or save WebVTT or TTML format files. 

Caption Maker was written in the IE10 timeframe, so it doesn’t support SDP, but it creates a good starting point. It only takes MP4 files, and not OGG or WebM. If you use IE10 or IE11, run Caption Maker, and press F12. You’ll be able to download the various files to experiment yourself.

Using track for other cool stuff

I teased it in the title, but you can get pretty creative by using JavaScript to fetch cues (captions) from your tracks. This article outlines in the series described above shows you some of the script things you can do. One thing is to created your own styled text. Since you can overlay a text field onto a video element, you can place text anywhere you want and style it with CSS. To be sure text lines up correctly, you’ll have to use the CSS position attribute. Once you have your layout, that’s not that hard.

This example uses the “metadata” as the kind of track. When you specify the track in HTML, you set the kind= attribute. Typically you set it to captions or subtitles, so the video element takes care of displaying the track text. However, by changing that to “metadata”, you now can get cues in JavaScript, but they don’t display on the screen. Here’s some code that takes captions and puts them in color (with the much hated Comic Sans font) at the top of the video element.

<!DOCTYPE html >
<html>
  <head>
  <title>Styled text example</title>
    <!-- only force Internet Explorer 10 standards for testing on local machine -->
  <style>
   
    #display
    {      
      position:absolute;   
      display:block;  
      text-align :center;
      color:yellow;
      font-size:24px;
      font-family:Comic Sans MS;
      text-shadow: 0.1em 0.1em 0.15em #333;
       z-index:100; /* set z-index to be sure div is on top */
    }   
   </style>

  </head>
  <body>
    <div id="player" style='width:640px;'> <!-- container div that is sized by script -->
       <video id="video1" controls style='width:100%;'>
        <source src="https://ie.microsoft.com/testdrive/Videos/BehindIE9ModernWebStandards/Video.mp4">
        <!-- by using "metadata" as the kind, it will suppress the video player's own caption display -->
        <track id="track1" label="English captions" src="entrack.vtt" kind="metadata" srclang="en" default>   
      </video>
      <div id="display">     
     </div>   
    </div>

    <script>
      var video;
        //  get objects associated with the video, track, and div elements 
        video = document.getElementById("video1");
        var disp = document.getElementById("display");
        var track = document.getElementById("track1");

        video.addEventListener("loadedmetadata", function () {
          document.getElementById("player").style.width = video.videoWidth + "px";  // make enclosure div width == video width
          disp.style.top = (video.style.top + (video.videoHeight * .05)) + "px"; // set the text to appear at 5% from the top of the video
          disp.style.left = video.style.left;  // set the text to appear relative to the left edge of the video
          disp.style.width = video.videoWidth + "px"; // set text box to the width of the video
        }, false);

        track.addEventListener("cuechange", function () {
          var myTrack = this.track;             // track element is "this"
           var myCues = myTrack.activeCues;      // activeCues is an array of current cues.                                                   
          if (myCues.length > 0) {
            disp.innerText = myCues[0].text;   // write the text
          }
         }, false);

    </script>

  </body>
</html>

 In this example, the metadata cues are a series of URLs synced to the video. When the guy shows the IE Testdrive site, you see the live site on the screen. This one is a little contrived, but you get the idea.

So that’s all there is to add accessibility and other cool features to your online videos. Give it a try.

The code uses the Media Source Extension API, and DASH files created with MP4Box, a packaging utility. There are a growing number of HTML5 and MSE based players showing how to do streaming. MSE lets you stream video to your page by substituting a buffer or buffers for a URL on the src property of a video element. Most of the samples I found on the web used a whole bunch of little files with segments of the video.

I read that you could also use a single segmented file. Using XMLHTTPRequest (XHR) and setRequestHeader with byte ranges,  you can get small chunks of video from a single video file. I heard it, but didn’t find a good example.. I’d always wanted to try out XHR and ranges, so I created a sample that uses MSE with a single file.

The article has two sections, building the sample, and a resource reference page. There’s also a working example to try it out. Note that the example only runs in IE11. It loads in Chrome, but I think it's having difficulty with the MP4 support.

Take a look and check out the article. Let me know what you think.

-Jay

Update:

I've rewritten the article and uploaded it to webplatform.org, and with the help of Renior at webplatform.org, uploaded the example. You can now play with the code in real time in the Dabblet environment. There are a few changes, fewer <divs>, and the video is stored locally on webplatform.org.

Enjoy.