CPU utilisation

CPU becomes a bottleneck at the point it becomes 100% utilised. Once you run out of spare CPU capacity, throughput and concurrency cannot increase anymore, so latency starts going up (it takes longer to get stuff done as more work is trying to share a fixed amount of CPU capacity). Historically this was resolved by making CPUs faster (reducing latency, which increases throughput). Nowadays it is more likely to be resolved by adding more CPUs (or cores) which doesn’t reduce latency but does increase concurrency and therefore throughput – assuming latency stays constant and the application can successfully distribute itself across multiple CPUs.

In theory each CPU core provides a concurrency of one (i.e. to process 100 concurrent operations you would need 100 CPU cores, as one core can only be processing one operation at any moment in time). However in practice this ends up not being the case because most operations don’t just need CPU and actually spend most of their time waiting for other (much slower!) resources like disk or network, which frees the CPU up to work on other things. It’s not uncommon for one CPU core to be able to handle hundreds of concurrent operations before it’s fully utilised, because over 99% of its time is spent waiting for other things to happen.

Making CPUs do large quantities of unnecessary work (like repeatedly executing the same compute intensive work rather than doing it once and caching it) reduces the CPUs ability to make itself free to process other operations and is a common cause of CPU bottlenecks.

In summary:

· CPU Speed determines the minimum latency.

· CPU utilisation determines the maximum throughput.

· Adding more CPU cores increases concurrency.

Memory capacity and speed

When memory runs out one of two things tend to happen – the application crashes or throughput falls and latency increases massively as memory is swapped out to (much slower) page files on disk. In some compute intensive applications with very low latency requirements (in the order of a few microseconds), the access time of main memory itself can start becoming a factor. However for most applications, memory is plentiful and fast and therefore rarely a bottleneck to performance – in fact caching resources in memory to avoid the higher latency or low throughput of other resources is the most common way of boosting application performance!

In summary:

· Memory access speed determines the minimum latency.

· Memory capacity determines the maximum throughput and concurrency.

Disk throughput and latency

Currently disk storage tends to be the poorest performing resource in most applications – it has the highest latency and lowest throughput of any component in a server. However it is improving all the time with the advent of Solid State Drives (SSDs) which almost eliminate the latency issues of mechanical drives (because they’re not actually drives, they’re just persistent random access memory) and generally have much better throughput as well.

Mechanical hard drives are much slower than people tend to realise and many applications suffer from bad performance because of a lack of understanding of the performance characteristics of hard disks. On average the time to access any data (latency) on a mechanical drive is about 5 milliseconds (this is the average time it takes for the drive head to move to the required track on the disk platter and the disk to spin to the required point on the track) which limits a hard drives throughput to about 200 read or write requests a second. This is partially offset by the fact that if the data you need is all located in one place on the drive, then the drive head doesn’t need to move and the data can be streamed onto\off the disk at over 20Megabytes a second (which is why your PCs hard disk can stream high resolution videos without stuttering).

High end systems reduce these fairly severe performance limitations by grouping together several hard disks into an array and effectively treating them as a single unit by accessing them in parallel, which increases concurrency and throughput and therefore reduces the average latency.

In contrast, the latency of a current, enterprise class SSD is measured in microseconds, it has a throughput that can exceed over 40,000 reads\writes per second and has streaming rates that can achieve over a Gigabyte per second. Ultimately SSDs will end up as fast as main memory at which point disk performance issues will become a distant memory, but until then, and especially while mechanical drives are still around, disk will remain the slowest resource in most systems.

In summary:

· Disk seek time determines the minimum latency.

· Disk read\write rates determine the maximum throughput.

· Putting several disks in an array increases concurrency.

Network throughput and latency

Within data centres, network performance tends to be pretty good where the servers are locally connected and it not uncommon to obtain throughput of 1 or 10 Gigabits per second with a latency of less than 1ms. That being said, applications still want to avoid making unnecessary network round trips between servers to avoid adding too much latency. For example, many applications are guilty of making far more trips to a database server than they need to, when all the data needed could be obtained in one or two database calls, avoiding lots of unnecessary network latency and making better use of the available throughput.

However both throughput and latency become a much bigger deal over the internet where consumers may have poor DSL lines that only download at 1 or 2 Mbit\sec (and upload at an even lower rate!) and there is an end-to-end latency that can easily be many tens of milliseconds.

Latency tends to get overlooked when talking about networks. Just because a network has a throughput of 1Gbit\sec doesn’t necessarily mean it’s going to be fast (remember from the pipe analogy - throughput doesn’t take into account how long it takes to get through the pipe, just the rate that it comes out the end). On the internet, it’s not uncommon for a request to have to pass through several routers, and possibly a couple of firewalls on its journey between the server and the final recipient of the data. Each of these hops adds latency and it could easily take tens of milliseconds for a packet to get from one end to the other. (I’ve just pinged www.microsoft.com in the US from my desk in the UK and the request passed through 12 routers and had a roundtrip of 144ms – which means it took 72ms for my request to get to the server in the US and another 72ms to get the response back).

The way many websites try to reduce the effect of this high latency is by making use of the fact that often only part of the available network bandwidth (throughput) is utilised. If a browser downloads one request at a time (i.e. has a concurrency of one) it might only utilise 10% of the available throughput, so by increasing the concurrency we can more fully utilise the available throughput, which in turn reduces the average latency. This is achieved by getting the browser to make several HTTP requests simultaneously.

For example, say it takes 12ms to download an image from a web site and this uses 2Mbits\sec bandwidth. If we download six images one at a time then it will take a total of 72ms and use 2Mbit\sec bandwidth. If instead we download all six in parallel then, assuming we have 12Mbit\sec bandwidth available, we can now download the same six files in just 12ms. Even if the available throughput is less than 12Mbits\sec we’re still better off – say we actually only have 8Mbit\sec bandwidth available, we will make full use of our available bandwidth and we will still download all the content in around 24ms - about 1/3^rd of the time compared to downloading one at a time.

In summary:

· Network bandwidth determines the maximum throughput and effective concurrency.

· Network length and intermediate devices determine the latency of travelling along the network.

Artificial resource limits

The first thing to clarify here is what exactly are “artificial resources”? These don’t necessarily have a physical manifestation but are usually software resources managed by an application which have an upper limit, or throttle, on their usage. Typical examples of these are:

· Locks to control concurrent access to an area of memory (e.g. critical sections, semaphores, events). These can affect throughput and latency significantly as they commonly reduce concurrency to one.

· Limits on the number of connections that can be made to a service (e.g. Database connections, TCP sockets).

· Limits on the creation of key resources (e.g. thread pools, limits on the number of active requests\transactions\operations).

All these artificially created throttles put a limit on the maximum concurrency that an application can support, and as we now know, if concurrency can’t increase, throughput will go down and\or latency will increase. These limits are normally in place for very good reasons: locks stop data from being corrupted. Other limits are because of design decisions (e.g. the number of possible TCP sockets is limited to 64K because it’s a 16 bit number) or are in place to protect a resource from hitting throughput limits when bad things might happen.

Overcoming concurrency throttling because of memory locks is generally one of the hardest of these problems to fix. It will require code changes and probably a redesign of the application to get rid of them. Often the best you can do is minimise the scope of the lock, to keep the time spent holding the lock to a minimum (reduce the locks latency and therefore increase the throughput).

Hard limits (like TCP sockets) are normally worked around in a couple of ways. The simplest is to scale out, to multiply the quantity of the resource available (increasing concurrency). The harder, but ultimately more efficient way, is to make better use of the available resource (e.g. reuse existing sockets rather than recycle, or put multiple sessions\requests over a single socket, which should increase throughput).

It’s the last case, where an artificial cap is in place to protect a resource which, in theory, should be the easiest to fix – because these limits are normally configurable and therefore all you should need to do is increase the limit until enough of the resource can be created so that it’s no longer a bottleneck. In reality it turns out that these can be some of the hardest performance issues to find, because:

· Often the limits are not obvious (they are system limits which you didn’t implement and aren’t even aware of), so it can be hard to find out what artificial limits even exist and what their impact is.

· The System doesn’t actually let you know that the limit has been hit. Commonly these limits get hit silently, with no tracing or logging being generated to indicate they have come into effect. All you know is that the performance isn’t very good for some reason. Quite often you can end up looking in the wrong place for the source of the problem, as it creates symptoms in other parts of the application that look like they might be the cause of the bottleneck, but aren’t!

· There are often several related limits that could be causing the problem, so even if you know which limits exist it can be hard to work out which one needs changing. (So you end up significantly increasing all of them, which gets rid of the bottleneck, but then you wonder why the limits were there in the first place…)

The first sign that you normally see that an artificial limit is being hit is when, regardless of what volume of traffic your application tries to process, you never hit a limit in one of the other major resources (CPU, Memory, Disk or Network) but latency gets worse as you try to increase the throughput. There is often no simple way of finding or resolving these issues other than lots of research, detective work and debugging. The ideal solution is to have good logging or monitoring which tells you that a limit is being hit and what it is.

In summary:

· Artificial limits are normally implemented by fixing the maximum concurrent access to a resource.

· The effect of them is that they put a cap on Throughput and therefore cause Latency to increase when that throughput limit is hit.

· They’re generally harder to find and fix than they should be!

So why does all this matter?

The fundamental problem is that the overall throughput of the solution is only as fast as the Primary Bottleneck. This can be demonstrated using our pipe analogy again. Imagine our pipe doesn’t have the same diameter along the whole of its length, but there is a narrower section in the middle…

It doesn’t matter how large the ends of the pipe are, water is only going to be able to flow from end-to-end as fast as it can get through the narrow constriction (the bottleneck!) in the middle, so the overall throughput and concurrency is going to be determined by the diameter of the narrow section, not the diameter of the end of the pipe.

So if an application is to have good scalability and remain responsive, every resource used by it has to be able to maintain high throughput and low latency as traffic volumes increase. The ability to stay highly responsive under load is limited by the Primary Bottleneck, but there are many things in large, complex applications which can cause bottlenecks and they all need to be understood and managed. It doesn’t matter if 99% of the execution path can support 10000 requests per second, if the other 1% only supports 10 - the application as a whole WILL only support 10 requests per second before response times start suffering and your customers start going elsewhere.

Finding the primary bottleneck is where performance testing comes in. You could try and guess where the primary bottleneck will be, but I can guarantee that at least 80% of the time you will guess wrong – and even if you guess right, you’ll have no idea at what level of load it’s going to start causing problems and what its impact will be. The only way to be sure where the primary bottleneck is, when it will hit and what its effect will be, is to put realistic load through the application, monitor it to see where problems start to occur and fix the problem to push the bottleneck further away. How you go about doing that will be the subject of future articles.

Written by Richard Florance

The first thing to understand is that ALL applications have a Primary Bottleneck - unless you have infinite resources you can’t avoid it. The Primary Bottleneck is the first resource in an application that hits a hard limit as you increase the volume of work you want it to process. The application will normally still work after it hits this bottleneck, but any further attempt to increase the volume of work you want it to process will normally just result in the time it takes to complete the work to increase.

Even massive web sites, like those run by Google and Amazon, have a Primary Bottleneck. If you manage to generate enough traffic against them they will hit their Primary Bottleneck and they will slow down - they have just done a really good job of making sure that the volume of traffic required to hit the primary bottleneck is so high that you will never normally be able to reach it. To find out what the Primary Bottleneck is and to be sure it isn’t hit until the application is handling a greater volume of traffic than it needs to support in peak use, you need to undertake performance testing and analysis to find and remove as many bottlenecks as possible. This article will discuss the causes of bottlenecks, how you do the testing and analysis to find and fix bottlenecks will be the subject of some future articles.

To understand what causes bottlenecks and why it is actually quite hard to create highly responsive applications that can also scale to handle high volumes of work, we need to understand exactly what latency, throughput and concurrency are and how they are interdependent. We’ll define these terms by using an analogy of water flowing through a pipe:

Latency

Latency is a measure of how long it takes an operation to complete and is normally measured in seconds. An operation could be an http web request, a database transaction, a message or any other measurable activity – an example of latency for a web site is “It takes 200ms to load a web page”. It terms of a water pipe, it’s how long it takes the water to get from one end of the pipe to the other:

Latency is affected by the length of the pipe (the shorter the pipe, the lower the latency) and the speed that the water is travelling along it (the faster the water is travelling, the lower the latency).

Throughput

Throughput is a measure of the rate that work can be completed and is measured in operations per second. An example of throughput for a web site is “The web site can process 500 pages per second”. Using the pipe analogy, it’s the volume of water that comes out of the end of the pipe every second:

Throughput is affected by the diameter of the pipe (a wider pipe allows a greater volume of water to traverse the pipe at any moment in time) and the speed that the water travels through the pipe (the higher the speed of the water, the more water can come out of the pipe every second).

Hold on! We can begin to see that there is a relationship between latency and throughput - they are both affected by the speed the water travels through the pipe. The thing that ties them together and allows us to define an equation to measure this relationship is…

Concurrency

Concurrency is the number of operations that are being processed at any moment it time. An example of concurrency for a web site is “The Web site can process 1000 http requests simultaneously”. Using the pipe analogy, it is the volume of water that is in the pipe at any moment in time and is therefore determined by the length and diameter of the pipe:

Interesting… so Concurrency has a relationship between both Latency (affected by the length of the pipe) and Throughput (affected by the diameter of the pipe) and it turns out there is a simple equation that defines this interaction:

Concurrency = Latency x Throughput.

If you want to dig into this more, this is actually known as “Little’s Law” and forms part of Queueing theory. As an example of how this works, if your web site has a requirement that HTTP requests have an execution time (latency) of 0.1 seconds and maintains a throughput of 1000 requests per second, then the application must be able to process 0.1 x 1000 = 100 concurrent active requests at any moment in time. If it can’t achieve this level of concurrency then the required latency and\or throughput CANNOT be met!

So we now know that for our application to achieve low latency and high throughput we must also support high levels of concurrency. Let’s look at the things that effect throughput, latency and concurrency in any application using current IT technologies.

Core system resources

The performance characteristics of any application is ultimately defined by its use of five resources:

1. CPU

2. Memory

3. Disk

4. Network

5. Artificial resources

The maximum concurrency (and therefore throughput and latency) is going to be constrained by whichever one of these resources hits its limit first and becomes the Primary Bottleneck in the application. In the next part of this article we’ll look at each of these resources, understand what determines their latency, throughput and concurrency and talk about what we try and do to avoid them becoming a bottleneck.

Written by Richard Florance

A couple of years ago I posted an article entitled 'Access to an ASP.NET website via multiple authentications' which I advise you to read if you want to make full sense of this new post. To briefly summarise for those short of time though, consider the scenario where you have a website deployed in Azure and in respect of securing access to the site:

"I want authN method X in production but in preview I want authN method Y followed by authN method X"

Out of the box Windows Identity Foundation (WIF) allows federation with Identity Provider X but does not support multiple federated sign-in such as "Identity Provider Y followed by Identity Provider X".

In the article I alluded to the fact that code existed for the scenario but I did not include it in the post; for various reasons it was not code I could simply make available. However, I did set myself the task of rewriting the entire stack of code in a more generic way so that it could be picked up by developers who wished to protect their website using multiple federated sign-ins.

Roll forwards two years and purely due to a colleague asking a question that skirted around federation, I span up my development VM and, before I knew what I was doing, the code samples were finished!

The code samples have been uploaded here. The rest of this article is a description of what is included and how it works.

Firstly though I have a confession to make. I envisioned that this solution would be possible using WIF 4.5 and the System.IdentityModel namespace but did not 100% confirm this to be the case. The scenario I knew that worked at the time was for WIF 4.0 and the Microsoft.IdentityModel namespace. i.e. – before WIF was baked into the .NET framework. So, the sample provided is for WIF 4.0 but I make a vow here and now to also get this working for WIF 4.5 in the very near future, if possible. Seeing as I have waited two years to get this far you can be sure that I will (eventually) research this.

Methodology

The approach is completely configuration driven by making sure the correct WIF modules are present in the web.config file for your site but preceded by a custom module I have called BouncerModule40 (Bouncer because the module acts like a nightclub doorman):

The order of these modules is critical because we want the custom code to run before out of the box WIF gets its hands on the request.

Multiple federated sign ins are driven by including one or more named WIF Service elements, and a custom configuration section, as shown below:

Custom config:

WIF config:

(Apologies for the place markers, I didn't want to give away the details of my Azure AD tenant)

The bouncer module will deal with each Service element in the order indicated by the custom identityProvidersToCall element. In this case, IDP0 will be used for authentication, followed by IDP1. After the bespoke authentications have succeeded, control is handed back to WIF proper and the unnamed Service element (at the bottom of the snippet above) is processed. This final Service element would ordinarily be the intended production sign-in for the site and ultimately produce the session cookie used to authorise all future access to the website. The bespoke authentications, after having been negotiated, are necessarily thrown away.

Conclusion

The scenarios that warrant the existence of this code are fairly niche but I have actually been asked where the code is a few times over the past two years. Whilst to properly exercise what I have given you will require the particulars of your own domain, I hope this code can be your starter for 10.

Good luck and no, I haven't forgotten my promise to try and make this work for WIF 4.5.

Written by Bradley Cotier

During development of an app the test team reported that a null reference exception was intermittently occurring. For a seasoned developer like myself I didn't think it would be too hard to repro the issue and in fact null reference exception candidates can often be identified simply by eyeballing the code. I had a stack trace and so knew which method to look in.

From the method in question, I thought I saw the culprit and the story went like this.

Say you have a class of some type:

And you use an instance of the type in a LINQ query:

All is well with the world. Now, what would happen when the following piece of code executes?

someFoo is null and so when the predicate is evaluated, a null reference exception occurs. Problem solved … right?

Wrong! This code does not cause an error. However, the following code does:

The only difference is that in the first instance the collection is empty, in the second example it is populated.

Retrospectively, the answer is obvious. Under the covers the predicate would be expanded into a foreach loop which, for an empty collection, would never get executed.

Conclusion

This post is mostly for interest sake and the conclusions to be drawn are not revelatory. Always check for nulls and don't assume you know it all.

Written by Bradley Cotier

This blog is intended to document our learning's about Miracast technology and explain things to consider when implementing a solution using Miracast. It is primarily about Windows 8.1 tablets and Windows Phone 8.1 as the authors work for Microsoft in the Worldwide Modern Devices Centre of Excellence (CoE). We work on a program called First Wave which assists market leading customers in using and deploying the latest Windows technologies. For video case studies around our projects go to http://www.microsoft.com/en-us/windows/enterprise/customer-stories/default.aspx .

The authors, Lutz Seidemann (Infra) and Paul Tallett (App Dev), are Solution Architects with Microsoft Consulting Services – Worldwide Modern Device Center of Excellence.

We worked on many global projects but the relevant one is the Mandarin Oriental Hotel Group , which is detailed in our second blog. 

FAQ

Why do I want Miracast?
Miracast technology allows you to wirelessly project your computer/tablet or phone screen onto an HDMI connected device such as a TV or projector. You can play movies for all the family to enjoy or simply enjoy a shared web browsing experience without the need for messy cables. It is similar in concept to Apple’s AirPlay

What is Miracast?
Windows and Windows Phone 8.1 added support for Miracast projection from most devices and can either use a “dongle” that plugs into your TV or nowadays some TVs come with Miracast built in. Often the dongle is USB powered so can plug into a spare USB port on your TV. Most of the dongles are about the size of a pack of cards but some low cost USB-stick-sized devices are starting to emerge.
http://www.amazon.co.uk/Actiontec-SBWD100A01-ScreenBeam-Wireless-Receiver/dp/B00KSAYMH4

Technically it was designed as a “wireless HDMI cable” so you can think of it that way.

What is the user experience like?
Plug your dongle into the TV and select the correct AV port on the TV and you will see a “Device ready for pairing” type message on the TV. On Windows 8.1 select Devices/Project from the Charms Bar and click on “Add a wireless display”. The computer will scan for devices in range and allow you to select one and go through the pairing process which may involve typing a PIN that is displayed on the TV screen. This is all rather similar to pairing a Bluetooth mouse or headset (but doesn’t actually involve Bluetooth).On the phone, go to Settings/”Project My Screen” and a similar scanning/pairing process will be initiated.

Once you have paired the device, it is persistent and you will not need to re-pair the device unless you pair the dongle with another device – you will simply be able to connect and display. You can pair with as many devices as you like and can unpair the device from Control Panel or Settings.

Note that once you log off or reboot your device, the mirrored connection is broken and cannot be re-established automatically. There are currently no public APIs to control Miracast projection.

What is performance like?
Performance varies with many factors – dongle manufacturer, computer CPU power, Wi-Fi traffic, Wi-Fi waveband etc (more details later) but you should be able to stream an HD movie from the Internet and project it to your TV with minimal stuttering. Note Miracast also transmits your sound channel as well as the picture.

What components are used in Miracast?
Miracast support is embedded into the display drivers, Wi-Fi drivers and device firmware (including dongle firmware) and it is quite sensitive to driver versions and revisions. The Miracast standard is constantly being tweaked and although backwards compatibility is promised, it is not always transparent. Each large Windows or phone update has required an update to the various components which may be on different delivery schedules. For example, getting a firmware update to the Miracast support in a TV is often a lengthy process and for this reason we recommend using a dongle rather than relying on the TV support. ActionTec have proved very responsive to requests and are tightly integrated into the Windows Operating System team (OSG) so they are the partner of choice for our team.

If you are experiencing problems, make sure you have all the latest drivers and hardware/dongle firmware installed.

What technology does Miracast use?
Miracast is an evolution of Intel’s proprietary Wireless Display (WiDi) technology and uses the Wi-Fi Direct protocol and therefore communicates directly between device and dongle without going through a Wireless Access Point or router. However, the first stage of establishing a Miracast connection senses if the computer Wi-Fi adaptor is already using a Wi-Fi connection with a router and uses the assigned channel for the Miracast communication to prevent thrashing in the Wi-Fi adaptor.

We have not seen dongles that allow the Wi-Fi channel to be manually configured so if the computer is moved and switches channel for signal strength reasons, the Miracast session may become choppy or even disconnect as it tries to switch Wi-Fi channel. Since the bandwidth requirement for transmitting HD video is very high, having more than 3 or 4 dongles sharing the same channel will probably see saturation. Also remember that if you are streaming video wirelessly from the internet, that the stream will be transmitted twice over the Wi-Fi network, one to the computer and once to the dongle. Google’s Chromecast differs in this respect as it can stream directly from the internet but only for certain applications.

What methods are available for pairing a device?

We have seen several methods of pairing the device:

• Random Pin. Dongle displays a random PIN during the pairing process which must be manually typed into the computer to pair. This prevents pairing with a device that you cannot see, for example a TV in an adjacent hotel room.
• Automatic pairing. Introduced on Windows in 8.1 Update 1 as the default, a device can automatically pair when you select it. Disabling this feature is sometimes desirable (eg hotel) but has to be done in the dongle firmware if it supports it but not from Windows.
• Fixed secret PIN. Some dongles support manually setting a secret PIN in the dongle firmware. This prevents the device from being ad-hoc paired to other devices (eg a hotel guest using his phone) and affords greater control. When pairing you will need to know what PIN has been set or you will be unable to use the device.
• NFC or QR code. Some devices are starting to emerge that use more convenient methods to pair than having to type a PIN. For example Nokia’s solution uses NFC http://www.wpcentral.com/hands-on-microsoft-screen-sharing-lumia-phones

How do I manage the dongle?
Different vendors implement various methods of performing management tasks for the dongle such as upgrading the firmware or changing settings such as device name or timeouts. Some use a local USB port where you load the firmware then press a button on the device to flash it, other use mini web servers on the device (like a router) and present an HTML interface so that they can be managed remotely. Some flavors of the ActionTec firmware use a custom SSID that you connect to in order to manage the device which is a problem if you have a lot of devices as they pollute the experience when trying to connect a laptop to a Wi-Fi network for example, as all the dongle admin SSIDs are visible. You may be able to set the admin SSID to hidden.

Do all Windows 8.1 and Windows Phone 8.1 devices support Miracast?
No. This requires both hardware (Wi-Fi adaptor) and device firmware support.

We have tested the following list of devices successfully: Surface Pro 1, Surface Pro 2, Surface Pro 3, Dell Venue 8 Pro, HP ElitePad, Nokia 630, 1020, 1520. Note these may require a firmware upgrade to add this support depending on the age of the device.

The following devices do not support Miracast: original Surface RT.

Android 4.2 (JellyBean) or better (eg KitKat) is needed for Miracast support. Apple devices do not support Miracast as they use AirPlay which is a competing proprietary solution (implemented on Apple TV).

Do all TVs and projectors support Miracast?
You almost always require an HDMI input on the TV, so devices that do not have HDMI can be eliminated. Most Miracast solutions use 1080p for their screen resolution, but many HD TVs only support 1080i or 720p. 720p is not really sufficient for Windows 8 as it is below the minimum resolution for Modern Apps so you can’t start a Modern app when running at that resolution. Some of the dongles (eg ActionTec) however, do support 1080i and the results can be very good. (P=Progressive, I=Interlaced, Progressive shows every line, Interlaced draws every other line then goes back and fills in the alternate lines). Picture quality is generally better on 1080p. Resolutions higher than 1080p (1920x1080) are not currently supported by Miracast so 4K for example is not supported.

We tested various dongles (Netgear, ActionTec, Belkin) and TVs with embedded Miracast support (Samsung, Philips, Sony) and arrived at the conclusion that the ActionTec dongle was the most flexible in terms of environment/features and gave the best performance. Also the engineering team were very responsive and turned around custom firmware versions in a few days. For this reason our team would recommend the ActionTec dongle.

Can I add a button to the Windows 8.1 Start Screen to project the screen to aid discovery?
There are no public APIs to control Miracast projection but attached is sample code that you can pin to the start screen that sends keystrokes to invoke the screen selection menu. The same menu is used for connection or disconnection so the app does not need any knowledge of whether the device is currently projecting or not.

The projection stutters or drops. How can I improve performance?
In a single device setting, make sure you have all the latest drivers and firmware releases including dongle firmware. If these still cause issues, you can experiment with older versions of the drivers as sometimes this helps

If you can set up a 5GHz Wi-Fi network rather than 2.4GHz (the default) and get your computers to use that by default, this will generally give better results. 5GHz has higher bandwidth but shorter range so your mileage may vary.

In a multi-device setting (e.g. hotel) network traffic may be an issue if multiple devices are using the same channel or if the network has a lot of traffic. Most Wi-Fi Access Points (AP) can be configured to prefer a certain channel, spacing these out across your environment in a matrix arrangement can considerably improve overall Wi-Fi performance. Using a signal strength meter to obtain hard data on signal strength is very important here. There are some signal strength apps available on smart phones if you don’t have a specialized device. Again using 5GHz can be advantageous although 2.4GHz networks can still provide good results if properly configured. Remember that although the Miracast communication uses Wi-Fi Direct and does not go via your router, the channel it uses is determined by the Wi-Fi channel being used by the computer and your connection may not automatically upgrade and switch if a better channel is available.

Note you may need to update your Access Point firmware in order to set up a 5GHz network or control channel separation.

Written by Lutz and Paul

Disclaimer: The information on this site is provided "AS IS" with no warranties, confers no rights, and is not supported by the authors or Microsoft Corporation. Use of included script samples are subject to the terms specified in the Terms of Use.

This blog is intended to document our learning's about Miracast technology and explain things to consider when implementing a solution using Miracast. It is primarily about Windows 8.1 tablets and Windows Phone 8.1 as the authors work for Microsoft in the Worldwide Modern Devices Centre of Excellence (CoE). We work on a program called First Wave which assists market leading customers in using and deploying the latest Windows technologies. For video case studies around our projects go to http://www.microsoft.com/en-us/windows/enterprise/customer-stories/default.aspx

The authors, Lutz Seidemann (Infra) and Paul Tallett (App Dev), worked on many of these projects but the relevant one is the Mandarin Oriental Hotel Group project which is detailed in the Customer Scenarios section. Both are Solution Architects with Microsoft Consulting Services – Worldwide Modern Device Center of Excellence.

What is Miracast

What is Miracast and how you can use it in your architectural concepts?

Let's have a first look to Wikipedia:
"Miracast is a peer-to-peer wireless screencasting standard formed via Wi-Fi Direct connections in a manner similar to Bluetooth. It enables wireless delivery of audio and video to or from desktops, tablets, mobile phones, and other devices. It allows users to, for example, echo display from a phone or tablet onto a TV, share a laptop screen with the conference room projector in real-time, and watch live programs from a home cable box on a tablet. Both the sending and receiving devices must support Miracast for the technology to work. However, to stream music and movies to a device, such as a TV, that does not support Miracast, adapters are available that plug into HDMI or USB ports. Miracast allows a portable device or computer to send, securely, up to 1080p HD video and 5.1 surround sound (AAC and AC3 are optional codecs, mandated codec is linear pulse-code modulation — 16 bits 48 kHz 2 channels).The protocol uses a direct Wi-Fi connection between the two devices without involvement of a wireless router and cannot be used to stream to a router access point. It was created by the Wi-Fi Alliance." (From <http://en.wikipedia.org/wiki/Miracast>)

Ok, here the same again in a nutshell:
Miracast is a protocol that will transmit audio and video between devices via Wi-Fi. It relies on the Wi-Fi network available (802.11n), using 2.4GHz and 5GHz bands. With this, you can enjoy multimedia content streaming from any Miracast enabled device such as your Windows 8.1 Tablet or Windows Phone to the Miracast device in our environment. It is not necessary that both devices are connected to the Internet. They only need to share the same local wireless network. The shared information is sent by the device via Wi-Fi through a Wi-Fi Direct connection to a receiver connected to the display device. The receiver then decodes the video signal and passes it to the TV display (or other display device). Miracast supports WPA2-PSK encryption, so all you share is safe.
Windows 8.1 or Android with version 4.2 and higher support Miracast natively. For Windows 7 you need to download the Intel WIDI drivers. If you familiar what Apple called Airplay, Miracast works similarly.
 
Miracast Devices
Broadcasting photos, videos, music and other media from your mobile device or PC to your TV need not be a laborious process, fraught with wires and extra peripherals. Miracast, and Intel's compatible cousin technology WiDi, let you beam whatever is on your device's screen right to a television, and there's a good chance that the devices you already own have this technology built in.

Nowadays, all Windows 8.1 PCs with recent Intel CPUs and Wi-Fi cards already have the hardware necessary. On the phone side windows 8.1 phones and Android 4.2 Jelly Bean or later you can use to send a Miracast signal. If you have a Miracast-capable Phone or Tablet, all you need is a receiver to attach to your home theater. To help you choose, have a look on current Miracast receivers.

Step 1.) Select your Miracast receiver
 
Rocketfish Miracast Video Receiver ($79.99):

Belkin Screencast 4.0.18.0 ($119.99): 

Netgear Push2TV PTV3000  ($59.99)

D-Link DHD-131 ($99.99)

Actiontec Screen-Beam Pro ($69.99)

Lumia HD-10 ($79.99)

You need make sure the Miracast device is compatible with your TV. Most Miracast solutions use 1080p for their screen resolution, but many older TVs only support 1080i or even only 720p. The only device we found that can support 1080i & 1080p is the Actiontec Screen-Beam Pro.

The Wi-Fi Alliance website maintains a comprehensive list of compatible devices, although that list does not account for Miracast-compatible technologies like Samsung's AllCast Share. More information, including a list of Wi-Fi Certified Miracast products, the Wi-Fi Alliance Display technical specification, white paper, and more is available at www.wi-fi.org/miracast.

Step 2.) Connect your Miracast receiver to your TV
All Miracast receivers plug into a TV's HDMI port and are ready to receive as soon as they turn on.

Step 3.) Connect your Windows 8.1 PC or Tablet 

After you added your Miracast connection , from now you can access via the usual project option.
 
For detailed instructions have a look here .

Step 3.a) Connect your Windows 8.1 Phone 
 

After you tap your phone is displayed to the TV

Step 3.b) Connect your Android Phone

Note: instructions differ depending on whether you are streaming from an different Android version. This is Android 4.4 (KitKat)

Go to Settings/Connections and tap on Screen Mirroring
 

You will then see this screen and you can tap your Miracast dongle:
 

The device will go through the pairing process:
 
And your screen is now mirrored and you can tap to disconnect:
 
 
Customer Scenario

One year ago Mandarin Oriental partnered up with Microsoft in early adopter program called Windows Firstwave. Mandarin Oriental’s intention is to make its wealth of online marketing content, hotel information and guest services more easily accessible to its digitally savvy guests. Leveraging the Microsoft platform, Mandarin Oriental is able to integrate a combination of services formerly offered via the television, telephone or guest directory with content available on the company’s website to deliver a more intuitive in-room experience for guests via custom touch apps running on Microsoft Surface Pro tablets. The Mandarin Oriental Group has piloted Microsoft Surface Pro tablets in four of their award-winning properties in London, Washington DC, Las Vegas and Tokyo.

Microsoft Consulting Services worked together with the hospitality solutions partner iRiS Software Systems, InterKnowlogy to develop a suite of custom Windows 8.1 touch apps, available in nine different languages, that manage services for in-room dining, concierge and housekeeping requests, a guest compendium which provides information about the hotel’s facilities and access to guest feedback and guest preferences.

Here a quick overview of the solution architecture before we go deep on our Miracast implementation experiences during the pilot:

During the Architecture design session we proposed to use Windows Embedded 8.1 Industry on Microsoft Surface Pro to Mandarin Oriental. This version of Windows 8.1 is functionally identical to the consumer/pro versions of Windows 8.1 with the exception that Embedded 8.1 industry adds capabilities to lock down the device - in this specific case a Write Filter is used to write all changes that happen to the OS after it is booted into a cache which is then overlaid on the file system. In normal operation this cache is transparent to the user and Industry Embedded 8.1 operates as any Windows 8.1 PC would. However when the Surface Pro is rebooted the cache is deleted restoring the Surface Pro device to the install state. This is how user state (files, downloaded apps, browsing history, customization, etc.) is removed through a reboot initiated at checkout. It's a simple and effective process and it's similar to the way that 3rd Party products (e.g. Deep Freeze) works except in this case, the capability is built into the OS making Surface Pro with Windows Industry 8.1 ideal for this type of commercial application. The user can install any app, download email, work on documents, change any configuration they want, including log into a Microsoft Account (MSA) with the surety that at checkout, or when the Surface device is rebooted, all user-specific data is removed.

Additionally, all OS updates are applied after checkout by automatically switching the device to Servicing Mode where the write filter is disabled and all the patches applied keeping the install image secure and up to date. For real-time updates required by application such as Defender or Microsoft Forefront you can add folder level write filter exclusions to prevent that data being wiped off at reboot. An important benefit of this approach is that specific hotel apps can be sourced from multiple developers, and these can be complemented by any app that can be downloaded from the Windows Store (the very functional Bing apps for example).

This contrasts with the alternative Android/iOS mechanisms in use for tablets deployed for this type of application in hotels. For these OS's which do not have such evolved management mechanisms, the typical approach is to build one large app that must contain all required functionality. The tablet is then locked in kiosk mode running this app only. Writing one big monolithic app is clearly a less efficient software development approach, and it removes from the user experience nearly all of the familiar tablet functionality associated with the OS - i.e. something as simple as a web browser - if such functionality is desirable to include in the user experience, must be written/included in the hotel app development process. In Windows 8.1 Industry Embedded such functionality, and much more, is included out of the box leaving the developers to spend their time and money developing line-of-business hotel-specific functionality which means they can be potentially much more productive.   (Read the Mandarin Oriental Hotel Group Press Release , Blog or watch the Video )

OK, let's circle back to Miracast, what this blog is all about :
Part of the Mandarin Solution is Miracast to provide all guests the possibility to stream there preferred videos, watch Hulu or Netflix, internet TV channels or any other content direct on the big in-room TV. Imagine you go into a hotel room and can watch your preferred movie without messing around with a cable connection or different kinds of adapters. You just connect to Netflix and watch, I would call this very slick.

What you need consider to build such a solution?
Remember that although the Miracast communication uses Wi-Fi Direct and does not go via your router, the channel it uses is determined by the Wi-Fi channel being used by the computer and your connection may not automatically upgrade and switch if a better channel is available.

In a multi-device setting network traffic may be an issue if multiple devices are using the same channel or if the network has a lot of traffic. That’s important for environments such as Hotels, where you have many Miracast devices (Dongle and PC/Phone) very close to each other.

Understand your Wi-Fi Network:
1. Most Wi-Fi Access Points (AP) can be configured to prefer a certain channel, spacing these out across your environment in a matrix arrangement can considerably improve overall Wi-Fi performance. Using a signal strength meter to obtain hard data on signal strength is very important here. There are some signal strength apps available on smart phones if you don’t have a specialized device.
2. In general the 5GHz bandwidth is less noisy than 2.4GHz ; you can find baby monitors, microwave or Bluetooth devices sending on 2.4GHz. More information here.
3. If you can set up a 5GHz Wi-Fi network rather than 2.4GHz (the default) and get your computers to use that by default, this will generally give better results. 5GHz has higher bandwidth but shorter range so your mileage may vary. Configure your Router for "Optimize for compatibility" mode, which allow to configure a separated 5GHz channel per room (36,40,44,48,149,153,157,161).

Drivers and Firmware
4. Make sure you have all the latest Wi-Fi drivers and device firmware releases
5. Update your Miracast dongle firmware.
6. Update your Wi-Fi Router firmware, especially if you plan to go for 5Ghz infrastructure.
7. Update your TVs when using the build-in Miracast function
 
 
Code samples
For the Mandarin Oriental project we wrote a Modern app that sent keystrokes to invoke the charms bar and bring up the projection screen which we pinned to the Start Screen to make it very easy for guests to discover this functionality. Since Modern apps are not allowed to send keystrokes to the system, we used a technique called Brokered Components which allows desktop code to be called from a side-loaded Modern app. Sending the keystrokes is fairly trivial, here's the code we used in the Brokered Component:

private async Task<int> StartProjectionTask(int n)
{
          await Task.Delay(1);
          InputSimulator.SimulateModifiedKeyStroke(VirtualKeyCode.LWIN, VirtualKeyCode.VK_K);
          Thread.Sleep(1000); 
          InputSimulator.SimulateKeyPress(VirtualKeyCode.DOWN);
          InputSimulator.SimulateKeyPress(VirtualKeyCode.DOWN);
          Thread.Sleep(1000);
          InputSimulator.SimulateKeyPress(VirtualKeyCode.SPACE);
          //Thread.Sleep(3000);
          //InputSimulator.SimulateKeyPress(VirtualKeyCode.UP);
          //InputSimulator.SimulateKeyPress(VirtualKeyCode.UP);
          //InputSimulator.SimulateKeyPress(VirtualKeyCode.UP);
          //InputSimulator.SimulateKeyPress(VirtualKeyCode.SPACE);
          //Thread.Sleep(1000);
          return 0;
}

Brokered Components in Windows 8.1 apps is documented in this article: http://msdn.microsoft.com/en-us/library/windows/apps/dn630195.aspx
And a step-by-step guide is here: http://devhawk.net/2014/04/25/brokered-winrt-components-step-one/

 
Summary
As a recap, Miracast is a very exciting technology and if you spend a little time on the key points it works very well. After testing a few dongles, we selected ActionTec because of the wider selection of TV resolution (1080i and 1080p) and configuration options we got in the firmware. Working with the Actiontec engineering team was a pleasure.

Your environment may differ, so we don’t want to give a generic recommendation. Those are the critical components :

• Device Drivers
• Dongle and Router Firmware
• Wifi Network configuration 
• Miracast Dongle

Finally, give us your feedback; share your implementation experiences; ask question and we will try to help.

Happy implementing! Lutz and Paul

Disclaimer: The information on this site is provided "AS IS" with no warranties, confers no rights, and is not supported by the authors or Microsoft Corporation. Use of included script samples are subject to the terms specified in the Terms of Use.

Enabling Trace in Azure Web Sites and Web Jobs

Let's quickly recap how to create trace statements and enable output in your web site or web job.

In your application you add relevant trace statements at strategic points, using an appropriate severity. This is achieved using the static methods on the Trace class as follows:

System.Diagnostics.Trace.TraceError("TraceError WebSite");
System.Diagnostics.Trace.TraceInformation("TraceInformation WebSite");
System.Diagnostics.Trace.TraceWarning("TraceWarning WebSite");

Once your application is deployed to Azure, you can enable output from the portal by going to the configure tab for the web site. The settings will apply to all web jobs running under that web site:

With the above settings, anything of verbose severity upwards will be logged to the file system and to Azure storage (table and blob). Clicking on the "manage …" buttons allow you to configure which table or blob container the output is written to. You really can't get simpler than that!

(Note: At the time of writing the new Azure portal currently allows you to only manage file system application logging. Managing application logging settings for table and blob is currently on the backlog and will be available in the near future. Alternatively you can use the REST API to apply these settings).

Here's what the trace output looks like in table storage:

Using the static Trace methods to generate such trace is fine for some applications but it has its limitations. For example, as shown in the above example you cannot specify an event id. Using these methods also prevent having finer grain control over what is output. For this reason, the TraceSource class was introduced to provide enhanced tracing features that can be used in place of the static Trace methods.

Using TraceSource in Web Sites and Web Jobs

When instrumenting an application it is sometimes desirable to provide tracing at different layers in the application architecture or for different components. For example, you may wish to have tracing specific to messaging, database access or business logic. Using TraceSource you can assign different TraceSource instances to those aspects of the application, and through configuration, control the behaviour of each TraceSource instance. I won't go into further details here but you can refer to the MSDN documentation for further reading.

In this example, I've chosen to add three categories of events to my web site for which I will add three different TraceSource instances:

public static readonly TraceSource Operational = new TraceSource("Operational");
public static readonly TraceSource Messaging = new TraceSource("Messaging");
public static readonly TraceSource Diagnostics = new TraceSource("Diagnostics");

Now I can generate events for each of the instances as required:

Operational.TraceEvent(TraceEventType.Warning, 101, "TraceEvent WebSite Operational");
Messaging.TraceEvent(TraceEventType.Information, 102, "TraceEvent WebSite Messaging");
Diagnostics.TraceEvent(TraceEventType.Verbose, 103, "TraceEvent WebSite Diagnostics");

One of the benefits of using a TraceSource is that you can assign an id to each event, to uniquely identify it in the output - crucial from an operations management perspective if you need to identify that a specific event has occurred.

In order to get TraceSource to work with Web Sites and Web Jobs, you need to be aware of three listener types that can be found in the Microsoft.WindowsAzure.WebSites.Diagnostics DLL. Currently these types are undocumented since the design goal has been to minimise configuration and typically you should not need to know about their existence. However, if you want to go off-piste and use TraceSource then you'll need to familiarize yourself with them because in your configuration you will have to reference the listener type in order to write events to the appropriate output:

In the Web.Config of your Web Site (or the App.Config of your Web Job), add the following configuration:

<system.diagnostics>

    <sharedListeners>
      <add name="AzureTableTraceListener" type="Microsoft.WindowsAzure.WebSites.Diagnostics.AzureTableTraceListener, Microsoft.WindowsAzure.WebSites.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      <add name="AzureBlobTraceListener" type="Microsoft.WindowsAzure.WebSites.Diagnostics.AzureBlobTraceListener, Microsoft.WindowsAzure.WebSites.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
      <add name="AzureDriveTraceListener" type="Microsoft.WindowsAzure.WebSites.Diagnostics.AzureDriveTraceListener, Microsoft.WindowsAzure.WebSites.Diagnostics, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35" />
    </sharedListeners>

    <sources>
      <source name="Diagnostics" switchName="DiagnosticsSourceSwitch" switchType="System.Diagnostics.SourceSwitch">
        <listeners>
          <add name="AzureDriveTraceListener"/>
        </listeners>
      </source>
      <source name="Messaging" switchName="MessagingSourceSwitch" switchType="System.Diagnostics.SourceSwitch">
        <listeners>
          <add name="AzureTableTraceListener"/>
          <add name="AzureBlobTraceListener"/>
        </listeners>
      </source>
      <source name="Operational" switchName="OperationalSourceSwitch" switchType="System.Diagnostics.SourceSwitch">
        <listeners>
          <add name="AzureTableTraceListener"/>
          <add name="AzureBlobTraceListener"/>
        </listeners>
      </source>
    </sources>

    <switches>
      <add name="DiagnosticsSourceSwitch" value="All" />
      <add name="MessagingSourceSwitch" value="All" />
      <add name="OperationalSourceSwitch" value="All" />
    </switches>

    <trace autoflush="true" indentsize="4" />

</system.diagnostics>

AzureTableTraceListener output

AzureBlobTraceListener output (csv file)

date,level,applicationName,instanceId,eventTickCount,eventId,pid,tid,message,activityId

2014-09-04T11:09:13,Warning,logging,09482d,635454257537318640,101,12792,52,TraceEvent WebSite Operational,

2014-09-04T11:09:14,Information,logging,09482d,635454257543880608,102,12792,52,TraceEvent WebSite Messaging,

AzureDriveTraceListener output

2014-09-04T11:09:14 PID[12792] Verbose TraceEvent WebSite Diagnostics

Using the Azure Web Site Logs Browser Site Extension

Site extensions are a great way of easily extending the functionality of your site through freely downloadable packages available from the site extensions gallery. One such site extension is the Azure Web Site Logs Browser which allows you to view your system and application logs (file, table and blob) through your web browser. Here's what the table storage content looks like:

So there you have it. Tracing in Web Site and Web Job applications using the standard .NET trace libraries is no different to any other .NET application. The key thing you need to know is what listener types you have available to you.

Happy tracing.

Written by Christopher Owczarek

Thanks to David Ebbo, Robert Jarratt and Amit Apple for their help in producing this post.

Overview

We encountered a challenge on our project where we wanted to play videos in MediaElement but the MP4 file was stored in Sharepoint which required user credentials for authentication. If you pass a URL to the MediaElement on Windows Phone 8.1 then it just tried to access the file without passing any authentication information and there is no easy way to intercept this. We escalated this to the Product Group and got the response that this was by design.

Solution

We came up with the idea of writing a custom IRandomAccessStream and setting it on top of WebClient so we could manipulate the http requests going out, and we wanted to use range headers so we could request just a sequence of bytes from the server and would not need to download the whole MP4 file in order to stream it. Although simple in concept, this proved quite challenging and took many complete rewrites of our solution and a team effort where we were all contributing ideas and code, but we finally got it working, so I’m posting it here.

There were three really tough problems –

1. Get the IRandomAccessStream implemented properly so that MediaElement would stream and play it
2. Get the http RangeHeaders working so that we didn’t need to download the whole file before it started
   playing
3. Figuring out the mix of IAsyncOperationWithProgress and the async/Task feature so we could do async
   operations in the middle of data requests

We actually got each of the three pieces working separately before combining them into the working solution. The web proved very little help as there seemed to be a lot of people struggling with this but nobody seemed to get it working so hence this blog post. We also saw people trying custom MediaStreamSource implementations but that looked much more complex than the custom IRandomAccessStream if we could get it to work.

The solution posted here uses Universal apps and has a Phone project and a Windows 8.1 project so works on both platforms. Most of the files in the solution are pretty much just the standard template files with the exception of StreamingRandomAccessStream.cs and the following lines in MainPage.xaml.cs:

protected async override void OnNavigatedTo(NavigationEventArgs e)
{
    var uri = new Uri("http://archive.org/download/RescuefromGilligansIsland/RescuefromGilligansIsland_512kb.mp4");
    var headers = await StreamingRandomAccessStream.GetHeaders(uri);
    ulong length = (ulong)headers.ContentLength;
    string mimeType = headers.ContentType.MediaType;
    var stream = new StreamingRandomAccessStream(null, uri, length);
    mediaElement.SetSource(stream, mimeType);
}

Here you see us fetching the headers from the MP4 file and creating a new instance of StreamingRandomAccessStream which is where the magic happens. Note that the sample Gilligan’s Island video does not actually need authentication but it was useful to test with as it is quite large. You’ll need to replace the dummy usernames and passwords in the sample to use the authentication piece.

The StreamingRandomAccessStream has some static methods for managing the http traffic but where it all happens is really this method:

public IAsyncOperationWithProgress<IBuffer, uint> ReadAsync(IBuffer buffer, uint count, InputStreamOptions options)
{
    return AsyncInfo.Run<IBuffer, uint>(async (cancellationToken, progress) =>
    {
        progress.Report(0);
        if (_stream == null)
        {
            var netStream = await GetStreamWithRange(_requestedUri, _requestedPosition, _requestedPosition + count);
            _stream = netStream.AsInputStream();
        }

        return await _stream.ReadAsync(buffer, count, options).AsTask(cancellationToken, progress);
    });
}

This method is called through the IRandomAccessStream interface by the MediaElement to request the bytes of the video. The really tough bit that took a lot of wrestling with was getting the delegate signatures right and figuring out how to do an await in the middle of an IAsyncOperation.

Note when using range headers on Windows Phone there's a slight difference to Windows 8 in that the end position cannot be null on phone otherwise it throws an error, despite being a nullable type. This works fine on Windows 8.

Anyhow, hoping this blog post helps you and thanks to the team – Iestyn Jones, Simon Middlemiss and Paul Tallett.

Enjoy!

VideoTest.zip

If you are lucky enough to have access and you are building a Windows Phone or Windows 8 application, then you can use this quick guide to allow you to track crashes in your apps.

Automatic exception logging in apps isn’t (yet?) a feature of Application Insights, but its perfectly possibly to use the in-build event logging functionality to do this.  To make this happen you need to have first setup Application Insights for your app.  I’m going to assume you’ve done this already as you’ve got this far.  If you haven’t then check out these articles on MSDN on how to do so for Windows Phone and Windows Store Apps and then come back here.

The logging API for Application Insights is really simple.

ClientAnalyticsChannel.Default.LogEvent("MyCategory/MyEvent");

This will log an event “MyEvent” that will be categorised under the title “MyCategory”.  You may have many event categories that you want to log across your app and you unhandled exception events will just be another category.  You can choose what you want to call it, but for this example we are just using ‘Exceptions’ as the category name. You’ll need to call this code from two places, the UnhandledException handler and the UnobservedTaskException handler although you can use a different category name to separate these out.

The event name is slightly more problematic. You want to have a unique event captured for each execution path that leads to an exception.  In order to do this you can generate a hash based on the stack trace of the exception.  Don’t forget that, in this world of async/await and AggregateException, you’ll need to include the child elements of the AggregateException and their InnerException as well.

The following method should give you a string that when hashed results in a unique id by which to classify your exception.

private static string GetFullDetail(Exception ex)

{

    string thisException = string.Format("{0}: {1} - {2}", ex.GetType().Name, ex.Message, ex.StackTrace);

    if (ex.InnerException != null)

    {

        // Recursively get inner exception details

        string innerExceptionDetails = GetFullDetail(ex.InnerException);

        if (string.IsNullOrEmpty(innerExceptionDetails))

        {

            thisException = string.Format("{0}\n\nINNER: {1}", thisException, innerExceptionDetails);

        }

    }

    var aggregateException = ex as AggregateException;

    if (aggregateException != null && aggregateException.InnerExceptions != null)

    {

        int count = 1;

        if (var aggregateInner in aggregateException.InnerExceptions)

        {

            // Don't include the aggregate that's just a duplicate of the inner.

            if (aggregateInner != ex.InnerException)

            {

                // Recursively get aggregate inner exceptions

                string aggregateInnerExceptionDetails = GetFullDetail(aggregateInner);

                if (!string.IsNullOrEmpty(aggregateInnerExceptionDetails))

                {

                    thisException = string.Format("{0}\n\nAGGREGATE({1}): {2}", thisException, count, aggregateInnerExceptionDetails);

                }

            }

            count++;

        }

    }

    return thisException;

}

By calling this method where you handle your exceptions you can use it as the event id when you call the logging code.

private static void HandleException(Exception ex)

{

    string exceptionDetail = GetFullDetails(ex);

    string hashCode = exceptionDetail.GetHashCode().ToString();

    Dictionary<string, object> dimensions = new Dictionary<string, object>();

    dimensions.Add("HashCode", hashCode);

    dimensions.Add("StackTrace", exceptionDetail);

    dimensions.Add("Type", ex.GetType().ToString());

    dimensions.Add("Message", ex.Message);  

    string eventMessage = string.Format("Exception/{0}/{1} - {2}", 

                                         Assembly.GetExecutingAssembly().GetName().Version.ToString(4),      

                                         detail.HashCode, 

                                         ex.Message);

    ClientAnalyticsChannel.Default.LogEvent(eventMessage, dimensions);

}

The method above is doing three things. 

1. Calling out first method to generate a unique and complete stack trace, we are then hashing this and using this as our identifier in 3.
2. Adding dimensions for StackTrace, Type and Message, along with the HashCode.  This allows us to extract the information we need to identify what is causing the crash.
3. Constructing the ‘eventMessage’ and logging the event with Application Insights.

You’ll see at the end where we construct the ‘eventMessage’ variable, the event log path is split into 2 categories and an identifying event.  Here we are further splitting the events down by the appVersion.  This makes it easier for us to focus on where the issues are in the current version of the app. Take a look at the screenshot below which makes this clearer.

Hopefully this gives a useful insight in to how to instrument your Store Apps so as to be able to track and fix the exceptions that are occurring in them.

I have multiple Azure Infrastructure-as-a-Service (Iaas) Virtual Machines (VMs) running under multiple Cloud Services, within one Virtual Network. I need these VMs to resolve each other based on hostname as well as be able to resolve public websites.

If these VMs were all running in one cloud service, they would be able to resolve each other using Azure Name Resolution. However, for a variety of reasons, I need to run them in separate Cloud Services, which prevents me from using this.

http://msdn.microsoft.com/en-us/library/windowsazure/jj156088.aspx

This leaves me with three options:

1. Use host files to record the names of each VM
2. Use an existing external DNS server
3. Create a DNS server

In this article, we'll explore the third option. Note this article may seem fairly basic to people who have a good understanding of infrastructure, but hopefully it will help people who don't.

Note: Azure AD is quite different from Windows AD and does not provide any functionality for DNS.

Azure Setup

To simulate this scenario, I have the following Azure setup…

Note, I actually have multiple VMs similar to Test1, each running in their own cloud service, but for simplicity I have just shown the DNS VM and another example VM.

Configuring DNS Virtual Network

If this is not already configured (as per the above scenario), go the Azure Management Portal and set a DNS server to the IP of your DNS VM.

Configuring DNS Server

To add the DNS server role, remote onto the Virtual Machine you intend to use for DNS and launch Server Manager. From this, launch the "Add Roles and Features Wizard" and do the following:

• Before You Begin Screen: Click Next on the first screen
• Installation Type Screen: Select the "Role-Based or Feature Based Installation" radio button. Click next.
• Server Selection Screen: Select your local server (DNSVM) then click Next.
• Server Roles Screen: Check the "DNS Server" tick box. When prompted to "Add features that are required for DNS Server?" click "Add Features" (ensure the management tools checkbox is also checked). Click Continue then click Next.
• DNS Server Screen: Click Next.
• Confirmation Screen: Click Install.
• Results Screen: View the results page to confirm installation has been successful, then click Close.

Once the DNS role is installed, to configure bring up the DNS Manager (in Server Manager, click DNS from the left hand pane, select your local server, right click and select DNS Manager)

Within DNS manager, you now need to create a forward lookup zone. This is the domain name on which all of your servers are referenced. In my example, I'm going to use contoso.local (local is to denote that the domain name is just pertinent to the virtual network, and not a public domain name – you could use whatever domain name you like however).

In the new zone wizard…

• On the first screen, click Next
• Select the type of zone to be a "Primary Zone" click Next
• Set the zone name to be your domain (e.g. contoso.local). Click Next
• Accept the defaults on the "zone file" screen, click Next
• IMPORTANT: On the dynamic update screen, ensure that the radio button for "Allow both nonsecure and secure dynamic updates" is checked. This is so new VMs can automatically register. Click Next.
• Review the settings and click Finish.

You should end up with the following

Setting the DNS Suffix on all VMs

The final configuration step is to ensure that all VMs (including the DNS VM) use a DNS suffix that matches your new forward lookup zone.

From the Server Manager, click "Local Server" in the left hand pane, then click the "Computer Name" link to launch the In the System Properties dialog.

• In the System Properties dialog, click Change
• In the subsequent "Computer Name / Domain Changes" dialog, click the More button.
• Finally set the DNS suffix to match the forward lookup zone (created on your DNS server). Click Ok to close dialog boxes and allow the machine to reboot.

Repeat on every Virtual Machine.

Verify Everything's Worked

To verify everything's worked, remote onto the DNS VM and launch the DNS Manager. Expand your forward lookup zone. You should see all of your VMs under this (the VMs would have automatically registered as we set our zone to "Allow both nonsecure and secure dynamic updates")

Finally, on each machine, launch a command prompt and run NSLookup against the other VMs in your network as well as a publicly facing website. These should now successfully resolve.

Footnote: By default, root hints are configured on the DNS server. This means you can resolve external sites also with no further action:

Written by Rob Nowik