If you want to build video applications on Windows 8 platform, you may find this blog useful. We (Windows Azure Media Services team) ships Smooth Streaming Client SDK for Windows 8 and Microsoft Media Platform Player framework (MMPPF) in addition to Windows 8 Media Foundation to help you build rich media applications.

Building Video application on Windows 8 - what are supported? 

If you are building a video application on Windows 8 and your video source is H.264 (.MP4) for instance:

• You could use MediaElement class in XAML application to handle the playback. Code sample: 

<MediaElement Source="Media/video1.mp4" AutoPlay="True" />

• Or you could use <video> tag in HTML5 application. Code sample: 

<video width="320" height="240" controls="controls"> <source src="movie.mp4" type="video/mp4"> </video>

Here is a detail list of video format that Windows 8 playback pipeline support (full details in Windows blog) : 

That's what happen when developing an application with .Mp4 URL for instance. Windows 8 Media pipeline has Decorder to decode the video, Renderer to render the image and Decrypter to decrypt the video if it is protected. As long as the video format you provides falls into the category I showed in the table above, Windows 8 Media pipeline will handle for you. Meanwhile, Windows 8 team put in a lot of efforts for optimizing all these operations to achieve battery optimization, hardware acceleration and etc.

Playing Smooth Streaming content using Smooth Streaming Client SDK for Windows 8

Read full blog here ..

What is Windows Azure Media Services

Figure 1: Windows Azure Media Services

Windows Azure Media Services offers a collection of components and technologies from Microsoft and third parties to enable end-to-end media solutions, leveraging on Windows Azure platform. If you are looking for end-to-end media solution with the power of cloud computing, Windows Azure Media Services is the right service you should consider.

Read the full article here. 

Sign up for the preview

1. Go to windowsazure.com and obtain a free-trail account or login with your own account.

2. Go to https://account.windowsazure.com/PreviewFeatures and that’s where you could request for feature preview. 

3. Click on Try it now showed in the picture above. After clicking it, you will see the following screen. Currently we offer three-months free trail and you could click on complete if you want to give it a try.

Continue reading here. 

This blog is a walk-through for building a “Hello World” version of Smooth Streaming Windows Store Apps, using JavaScript and HTML5. This sample application uses Smooth Streaming Client SDKs directly. But we also recommend you to take a look at Microsoft Media Platform Player Framework for Windows 8, which is a player framework built on top of client SDKs. It is very easy to use for you to build rich video applications. If you are using earlier version of Client SDK, you could refer to this blog.

Continue reading here.

This blog belongs to a series of Smooth Streaming Client v2.0 walkthroughs. You could download this latest Smooth Streaming Client SDks here. We have made a few major changes in this version. In this blog, I will introduce to you how to figure out data chunk information while you are downloading it. 

While playing smooth streaming content, you could utilize ChunkDownloadedEventArgs to figure out downloaded chunk information, such as whether data chunk is successfully downloaded, what type of data chunk is, what duration the chunk has and etc. This is very useful for you to monitor data chunk download progress. If you find something wrong, you don’t have to wait the downloader to keep retrying, rather you could route to other media source for instance.

Here is all the information you could get from ChunkDownloadedEventArgs:

 public class ChunkDownloadedEventArgs : EventArgs
 {
 public Uri CanonicalUri { get; }
 public ChunkInfo Chunk { get; }
 public ChunkResult.ChunkResultState DownloadResult { get; }
 public ChunkRequestType RequestType { get; }
 public HttpStatusCode StatusCode { get; }
 public TrackInfo Track { get; }
 }

You could refer to the table below for more information.

The way to implement ChunkDownloadedEventArgs is quite straight forward. Firstly, we need to hook up ChunkDownloadedEventArgs with smooth streaming player. 

SmoothPlayer.ChunkDownloadFailed += new EventHandler<ChunkDownloadedEventArgs>(SSME_ChunkDownloadFailed);

Here I implement a method to read out all the useful information from the event args.

 private void SSME_ChunkDownloadFailed(object sender, ChunkDownloadedEventArgs args)
 {
 {
 statusTxt.Text = "The actual request type is " + args.RequestType+"\n";
 if (args.Chunk != null)
 {
 statusTxt.Text += "chunk is not null"+"\n";
 statusTxt.Text += "chunk duration is " + args.Chunk.Duration + "\n";
 statusTxt.Text+="chunk time stamp is "+args.Chunk.TimeStamp+"\n";
 statusTxt.Text += "chunk stream is " + args.Chunk.Stream + "\n";
 }
 
 if (args.Chunk.Stream != null)
 {
 statusTxt.Text += "chunk's stream is not null"+"\n";
 }
 statusTxt.Text+="This chunk is "+ args.Chunk.Stream.Type+"\n";
 if (args.Track != null)
 {
 statusTxt.Text += "track is not null"+"\n";
 }
 statusTxt.Text+="The result is "+ args.DownloadResult+"\n";
 statusTxt.Text += "The result is " + args.StatusCode+"\n";
 };
 }
 

1. You will need to add the player in .xaml file. I didn't set the streaming source in SmoothStreamingMediaElement in .xaml file.

<SSME:SmoothStreamingMediaElement AutoPlay="false" x:Name="SmoothPlayer" Grid.Row="0" />

2. And go to the code behind, and insert the following code

 public MainPage()
 {
 InitializeComponent();
 
 SmoothPlayer.SmoothStreamingSource = new Uri("http://devplatem.vo.msecnd.net/Sintel/Sintel_VC1.ism/manifest");
 
 SmoothPlayer.MediaOpened += (o, e) =>
 {
 SmoothPlayer.Play();
 };
 }

You will see from the code above, the first step is straight-forward, that we assign the media source URI to SmoothStreamingSource. However, if now, you immediately call SmoothPlayer.Play(), you will get an error. That's because when media source gets set up, Smooth Streaming player will do a bunch of things, such as downloading manifest and downloading some amount of data chunks. Thus, initially I called SmoothPlayer.Play() after ManifestReady() event, that's also wrong. Because Smooth Streaming player still needs to validate the media stream and read the file reader. However, MediaOpened event guaranteed all these. For details, you could read on MediaOpened() MSDN documentations here. 

In addition, if you want to read more about how we design different state of the media elements, you could read MSDN documents Media Element State Transitions.  

The IIS Smooth Streaming Client provides the ISmoothStreamingCache interface to support offline scenarios. Here is how it works in high level. 

When user starts to play video, Silverlight Smooth Streaming Client will check with Isolated Storage Instance (this isolated storage is within Silverlight player) to see whether there is video manifest or data chunks available to read. If not, Isolated Storage Instance will return a NULL as CacheResponse. Now, client knows there is no cache available for retrieving, then, it will go ahead to talk to server for getting manifest/data chunks, and stored it in the cache for next-time use. Here is a high level diagram for this conversation.

Second time, since we have saved data into cache (isolated storage), when client check with cache using BeginRetrieve(), cache will give back the data in EndRetrieve(). Client won't need to make a request to server in this case. 

Here is a detailed documentation on how to implement Cache Plugin-in on MSDN. 

I did a simplified version for demo purpose and you could download it here. And let's explain the code in steps.

0. This project is based on Smooth Streaming Client 1.5. For setting up instruction, please refer to my another blog post. 

1. In MainPage.xaml, a smooth streaming player is created: 

 <SSME:SmoothStreamingMediaElement AutoPlay="false" x:Name="SmoothPlayer" SmoothStreamingSource="http://mediadl.microsoft.com/mediadl/iisnet/smoothmedia/Experience/BigBuckBunny_720p.ism/Manifest" Grid.Row="0" />

2. And in MainPage.xaml, there are only "Play" button, "Stop" button and "Clear Cache" button.

3. Open MainPage.xaml.cs, and here are all the methods listed in this file. (Click to view a larger image)

4. If you open SmoothPlayer_Loaded(obejct sender, RoutedEventArgs e) method, you will see: 

 void SmoothPlayer_Loaded(object sender, RoutedEventArgs e)
 {
 cache = new ISO_StorageCache();
 SmoothPlayer.SmoothStreamingCache = cache;
 }

Basically, when ISO_StorageCache getting initialized, we will create a Isolated Storage file for storing manifest and data chunks. The URL/file name pairs stored in the IsolatedStorageSettings.ApplicationSettings object are read into a dictionary object when the cache is instantiated, and the dictionary pairs are used to identify data files in the cache. Here is the code:

 public ISO_StorageCache()
 {
  IsolatedStorageFile isoFileArea = IsolatedStorageFile.GetUserStoreForApplication();
 
 foreach (System.Collections.Generic.KeyValuePair<string, object> pair in IsolatedStorageSettings.ApplicationSettings)
  {
  if (!keyUrls.ContainsValue((string)pair.Value) && isoFileArea.FileExists((string)pair.Value))
   KeyUrls.Add(pair.Key, ((string)pair.Value));
  }
 }

5. Still looking at SmoothPlayer_Loaded method, "SmoothPlayer.SmoothStreamingCache=cahe". In this line,an ISmoothStreamingCache object is started and a request for data is issued, the Silverlight Smooth Streaming Client will call each of the methods in the order: BeginRetrieve, EndRetrieve, BeginPersist, EndPersist as showed above. And I have explained the use of these four methods at the beginning. 

6. At the end, if you want to modify the result(manifest or data chunks) before going into cache, you could catch the CacheResponse before it is saved into Isolated Storage file in EndPersist() method. CacheResponse will give you information for your content type, a response(manifest or data chunks) in stream format and header.

7. I often hit the problem that isoFile size is not enough and you could increase isoFile's size in EndPersist() by adding the following line. Or you could press the clean cache button. 

 /*Increase ISOFileArea size*/
 long availbleCacheBytes = isoFileArea.AvailableFreeSpace;
 long saveQuotaNumber = isoFileArea.Quota;
 
 isoFileArea.IncreaseQuotaTo(isoFileArea.Quota + 1048576);

• IIS Smooth Streaming Client 1.5 APIs : developers could utilize these APIs to build applications on Silverlight platform (runs on desktop,Xbox or Windows phone). These APIs is formally known as "Smooth Streaming Player Development Kit
• Microsoft Media Platform - Player Framework v2.6 : a open source framework. It includes sample player and numbers of plugins for developers to get quick started. And MMPPF is based on IIS Smooth Streaming Client APIs. 

 

3.  Now you could see two projects are generated in Solution Explorer. PlayDemo1 is the Silverlight application with XAML files. And PlayerDemo1.Web is a web application project with HTML page, which   contain the Silverlight component you built in PlayerDemo1. If you build this application now, you will see a PlayerDemo1.xap get generated in ClientBin under PlayerDemo1.Web project. 

4. Right click on References fold under PlayerDemo1 project, choose Add Reference...

5. When you install Smooth Streaming APIs, the APIs are resided at /%SystemDrive%/Program Files(x86)/Microsoft SDKs/IIS Smooth Streaming Client/v1.5/Silverlight. And choose Microsoft.Web.Media.SmoothStreaming.dll. 

6. Add namespace in MainPage.xaml under PlayerDemo1 project.  

xmlns:SSME="clr-namespace:Microsoft.Web.Media.SmoothStreaming;assembly=Microsoft.Web.Media.SmoothStreaming" 

7. Add XAML for the media player and the whole MainPage.xaml should look like this:

<UserControl x:Class="PlayerDemo1.MainPage"
 xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
 xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
 xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
 xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
 xmlns:SSME="clr-namespace:Microsoft.Web.Media.SmoothStreaming;assembly=Microsoft.Web.Media.SmoothStreaming" 
 mc:Ignorable="d"
 d:DesignHeight="640" d:DesignWidth="480">
 
 <Grid x:Name="LayoutRoot">
 <SSME:SmoothStreamingMediaElement AutoPlay="True" x:Name="SmoothPlayer" SmoothStreamingSource="http://streams.smooth.vertigo.com/elephantsdream/Elephants_Dream_1024-h264-st-aac.ism/manifest" Grid.Row="0" />
 </Grid>
 </UserControl>

8. Lastly, press F5 and you will compile the project and see a simple smooth streaming client. 

• Hurl: you can download at http://hurl.it/

Hurl is a Web-based tool for testing APIs. You could choose the request method, customize headers and POST parameters, add basic authorization, and follow redirects. Then view the nicely formatted request and response. It is a very clean and easy tool for testing your APIs.

Here is the Twitter API reference page I used: https://dev.twitter.com/docs/api/1/get/search

HURL provides five HTTP methods: PUT, GET, POST, DELTE, and HEAD. And you could apply HTTP basic access authentication. You could also view request and response body in full size view (a pop up window). Another thing I like Hurl is that for response body, it highlights the contents with different colors, which made it really easy to read.

You could also login and after that, you could view your call history:

By clicking on each URL you will be redirected back to request page with parameters you used previously. And you could repeat the test again or modify on the past test.

• Fiddler: you can download at http://www.fiddler2.com/fiddler2/

  Fiddler is a free Web Debugging Proxy which logs all HTTP(S) traffic between your computer and the Internet. It is a desktop application only runs on Windows Operating System.

There are two parts of Fiddler you may be interested in immediately. Firstly, it is the request builder showed above. You could customize your request. It has almost all types of HTTP methods supported. By pressing the Execute button you will see the response return.

Secondly, it is the Session Inspector. By clicking on the web sessions history on the left side, you could inspect each session individually with details showed up on Inspectors. 

Another interesting feature for Fiddler is that it provides Web View, so you could see images that are returned.  This allows you to see specifically what was returned to the browser before it was interpreted.

Fiddler is a very powerful tool has much more functions than I introduced. You could refer to Fiddler help document for more information. http://www.fiddler2.com/Fiddler/help/

Currently, HTML5 audio element has five attributes:

• autoplay: Specifies that the audio should start playing as soon as it is ready.
• controls: Specifies that playback controls should be displayed.
• loop: Specifies that the audio should start over again, when it is finished.
• preload: Specifies whether or not the audio should be loaded when the page loads.
• src: Specifies the URL of the audio to play.

This is how to implement HTML5 video tag with one single line of code.

 <audio id="player" src="YOUR_SOURCE_HERE" controls="controls" autoplay></audio>

And you can see the UI rendered like this in IE9. It shows a +30s and -30s when you mouse over the timer on both side. Also, current time and time for the whole song will be displayed as well.  

Same Audio tag rendered like this in Chrome 11. It doesn’t show much as IE9, such as overall timing and 30s fast-forward/fast-backward.

This is how Audio Tag looks like on Safari 5. There is no current time and total time. However, you can drop to next song/previous song.

I tired Opera 11 as well. It rendered in the browser but it doesn’t play.Same thing happend to Firefox 4 Beta. Later on i found out it was because they don't support the audio format .Mp3. 

I further tested Audio tag on iPhone 4.3 and iPad 4.3.3. It rendered as below. Moreover, when you minimize browser, music will still continue in the background. Double tab you could open the multi-task bar at the bottom of screen, swap right, you could see the native audio player control. I truly like this kind of native integration with HTML5 tag.

I tested Andiord phone as well, both 2.3 and 2.2 version. The Audio bar doesn’t get loaded. Current Windows Phone 7 doesn’t support Audio tag. However, since IE9 will be shipped with next version of Windows Phone 7, obviously, audio tag will be native to Windows Phone 7 browser in the near future as well.

I had a problem with starting a song with a specific timing, such as starting a song from 20 seconds (streaming). Because the song may not be loaded till that point, so if you jump to that point, it will have javascript error. Here is a way to solve it. And you can definitely improve it.

<!DOCTYPE html>

<html>

<head>

<script type="text/javascript">

    function timeMsg(startime) {

        var t = setTimeout("a("+startime+")", 5000);

    }



    function a(startime) {

        $("#player")[0].play();

        $("#player")[0].currentTime = startime;

    }

    function song_onClick(item) {

        var startTime = item.id;

        var songSrc = item.title;

        alert(songSrc + startTime);

        $("#player")[0].pause();

        $("#player")[0].src = songSrc;

        $("#player")[0].load();

        timeMsg(startTime);

     }

 </script>

</head>

<body>

 <audio id="player" src="YOUR_SOURCE_HERE" controls="controls" autoplay></audio>

    <div class="song" title="YOUR_SOURCE_HERE" id="20" onclick="song_onClick(this)">

        Maid with the Flaxen Hair

    </div>

</body>             

</html>

You can see from the code, the ID of the audio element is player.

$(“#player”)[0].src = songSrc; is used to change the song src. So that we can change to the next song.

However, here is the track. If you wanted to start a song from some random point, you have to load the song first (if the song is streamed). Hence, I used setTimeOut method here to delay playing the song for 5 sec and I started play the song from 20 seconds. However, still, if you start more than 20 seconds, 5 seconds may not be long enough. You could design it to times a multiplier.

Hope this post will inspire you to try out your audio application in HTML5, so you could benefit from one code base works for most of browsers and devices!