Keynote started with a video of developers, designers etc. working on Windows 8 giving their favorite features in Win8.

• ~450 million copies of Win7 sold (1500 non-security product changes seamlessly delivered)
• Consumer usage higher than XP
• 542 million Windows Live sign-ins every month

Lots of change in Windows

• Form factors/UI models create new opportunities (touch)
  • "People who say touch is only for small or lightweight devices are wrong. As soon as you use touch on a tablet, you're going to want to touch on your desktop & laptop."

• Mobility creates new usage models – e.g. use while reclining on a couch

• Apps can't be silos - "customers want a web of applications"
  • Apps to interact easily
  • Services are intrinsic

What is Win8?

• Makes Windows 7 even better - everything that runs on Win7 will run on Win8

• Reimagines Windows from the chipset (ARM work) through the UI experience
  • All demos shown today are equally at home on ARM and x86

Performance / Fundamentals

Kernel Memory Usage

Demos

User Experience (Julie)

1. Fast and fluid - everything's animated
2. Apps are immersive and full screen
3. Touch first - keyboard/mouse are first-class citizens ("you're going to want all three")
4. Web of apps that work together - "when you get additional apps, the system just gets richer and richer"
5. Experience this across devices and architectures
6. Notes from Julie's demo

• Picture password - poke at different places on an image (3 strokes) to login

• Tiles on the home screen - each is an app - easily rearranged. Pinch to zoom in/out

• On screen keyboard pops up

• Swipe from right side to bring up Start screen - swipe up from bottom to get app menus ("app bar") - relevant system settings (e.g. sound volume/mute) also appear

• Select text in a browser - drag from right side to see "charms" - these are exposed by apps. One is "Share" - shows all apps that support the "Share contract".
  • Think of sharing as a very semantically rich clipboard.
  • Target app can implement its own panel for information (e.g. login, tags, etc.) for sharing when it's the target.

• Search
  • Can search applications, files - apps can also expose a search contract to make it easy for search to find app-specific data.

• Inserting a picture
  • Shows pix on computer
  • Social networking sites can add content right into picture file picker

• Showed settings syncing from one machine to another machine she is logged in on that is an ARM machine.

Metro-style Platform/Tools (Antoine)

• Current platform a mixed bag - silo of HTML/Javascript on top of IE, C#/VB on top of .NET & Silverlight, and

• Metro apps can be built in any language

• Reimagined the Windows APIs - "Windows Runtime" (Windows RT).
  • 1800 objects natively built into Windows - not a layer.
  • Reflect those in C#/VB.Net/C++/C/JavaScript
  • Build your UI in XAML or HTML/CSS

• Launch Visual Studio 11 Express - new app to build Metro apps.
  • Pick the language you want - pick the app template you want.

• Enable millions of web developers to build these apps for Windows.

• Code you write can run either locally or in a browser from a web server - just JavaScript and HTML 5.

• New format - App Package - that encapsulates

• Use mouse or touch seamlessly - no special code.

• Modify button to bring up file picker dialog…
  • Also allows connecting to Facebook if the app that connects FB photos to the local pictures is there - every app now gets access to FB photos.

• Adding support for the "Share" contract is 4 lines of JS

• Use Expression Blend to edit not just XAML but HTML/CSS.
  • Add an App Bar - just a <div> on the HTML page.
  • Drag button into there to get Metro style where commands are in the app bar

• Uses new HTML 5 CSS layout as Grid. Allows for rotation, scaling, etc. Center canvass within the grid.

• Expression lets you look at snapped view, docked view, portrait, landscape.

• 58 lines of code total

• Post app to the Windows Store
  • In VS Store / Upload Package…
  • Licensing model built into app package format. Allows trials.
  • Submit to Certification
    • Part of the promise of the store to Windows users is the apps are safe and high quality.
    • Processes can be a bit bureaucratic.
    • Does compliance, security testing, content compliance.
    • Will give Developers all the technical compliance tools to run themselves.
  • The Store is a Windows app. Built using HTML/JavaScript

• Win32 Apps
  • Not going to require people to rewrite those to be in the store.
  • Don't have to use Win8 licensing model.
  • Give the Win32 apps a free listing service.

• XAML / Silverlight
  • Using ScottGu sample SilverLight 2 app.
  • Not a Metro app - input stack doesn't give touch access.
  • How to make it a Metro app?
    • Runtime environments between SL and Win8 are different.
    • Had to change some using statements, networkin layer.
    • Reused all the XAML and data binding code - it just came across.
    • Declare it supports "Search" and add a couple of lines of code.
  • Also can use same code on the Windows Phone.
  • "All of your knowledge around Silverlight, XAML just carries across."

• If you write your app in HTML5/CSS/XAML, it will run on x86/x64/ARM. If you want to write native code, we'll help make it cross-compile to these platforms.

• IE 10 is the same rendering engine as for the Metro apps.

• Can roam all settings across your Win8 machines - including you app settings if you want.

Hardware Platform (MikeAng)

• 8 second boot time - win7 pc.

• UEFI

• New power state called "Connected Standby"
  • Windows coalesces all the timer and network requests, turns the radio on periodically to satisfy them, then goes back to very low power consumption.
  • But because app requests are getting satisfied they are up to date as soon as you press "ON"

• USB 3 ~4x faster at copying a 1 GB file than USB 2
• Can boot Win8 from up to 256 TB drive.

• Direct Compute API - can offload compute loads to GPU

• Every Metro app has hardware acceleration UI baked in.

• Doing work with OEMs on testing sensitivity of touch hardware
  • Windows reserves only one pixel on each side for the Windows UI, so sensitivity important.

• Down to 1024 x 768 for Metro apps. If 1366 x 768, get full Windows UI (side-by-side snap in). Any form factor - about resolution.

• Have a sensor fusion API - accelerameter, touch.

• NFC - near field communication - business card can have a little antenna built in to send data to Win8.

• Integrating device settings (web cam, HP printer, etc.) into Metro UI rather than as a third-party app.
• Ultra Books
  • Full core powered processor in a super-thin and light package.
  • Some are thinner than legacy connectors - RJ45 and VGA - they are bumps.
  • These things are mostly battery.

• Samsung PC giveaway - to all BUILD attendees
  • 64 GB SSD
  • 4 GB RAM (Steven: "so you can run Visual Studio")
  • AT&T 3G included for one year (2GB/mo)
  • Windows tablet + development platform.
  • 2nd generation core i5
  • 1366x768 display from Samsung - amazing

• Refresh your PC without affecting your files
  • Files and personalization don't change.
  • PC settings are restored to default
  • All Metro apps are kept - others are removed.
  • Command-line tool to establish base image for this for pros.

• Hyper-V in the Windows 8 client

• ISOs get mounted as DVD drives.

• Multi Mon -
  • Screen background extends
  • Task bar customizes to multi-mon - can have identical across two mons or have per-monitor task bar (show only apps running on that monitor)
  • Ctrl/PgDn to switch Metro start screen between the two monitors - develop on one, test on another.

• Keyboard works the same - type "cmd" from Metro Start screen and are in search for CMD.

Cloud Services (ChrisJo)

• Windows Live mail Metro client connects both Exchange and Hotmail.
  • Full power delivered by ActiveSync.

• Windows Live Metro calendar app.

• Bring together all the Friends through Linked In, Facebook, Windows Live.

• Photos
  • Connected to Facebook, Flickr, local photos.
  • Written as a Metro app.

• SkyDrive - 100 million people.
  • Every Win8 user, every Win Phone has a SkyDrive.
  • Also accessible to developers - access the same way as you would use local store.

Wrap

• Used college interns to develop sample apps included in dev preview build.

• 17 teams (2-3 devs per team).

• 10 weeks.

Developer Preview (not Beta).

Learn more:

• http://blogs.msdn.com/b8

MSFT will let everyone download the Developer preview starting tonight.

http://dev.windows.com

• X86 (32- and 64-bit)

• With Tools + Apps or just Apps

• No activation, self-support.

I'll be posting here my impressions from the keynotes and from the sessions I attend.  There is a rumor that a new tablet with Win8 is going to be distributed to attendees - we'll see.  Microsoft gave an Acer tablet to all attendees at the last PDC I attended in 2009.  Looking forward to getting my hands on a beta of Win8 and especially comparing it to my current favorite tech device, the iPad2 I bought in June.  There are some promising indicators - a friend sent me a link to this video showing an absolutely amazing bootup time for Win8.  We'll see...

I downloaded the BidNow sample and followed the directions under Getting Started on the home page to install the various required components – this required me to update my Azure tools to the release from the October 2010 PDC (it’s been a while since I’ve played with Azure) and the AppFabric SDK as well as the Identity Foundation SDK and runtime.

There are a set of automated scripts the developers of BidNow have provided to configure things, which is good and bad.  It’s good in that the configuration is somewhat complex and the scripted PowerShell scripts take you through setting it up by asking you a set of questions, TurboTax-style.  It’s bad in that error-handling in the scripts isn’t very robust and so a number of times I encountered stack traces and exception messages from PowerShell which quickly disappeared from my console as the PowerShell scripts blithely continued running, clearing the screen of previous errors before I could even read them.  It’s too bad that PowerShell doesn’t have an easy way of comprehensively logging everything that happened so I could go back and review which steps had failed so I could try to do those manually.  For instance, here is one error I managed to capture:

It’s unclear whether the error is expected since a Yahoo provider didn’t exist, or whether the error occurred trying to set up the ACS.  It’s also unclear whether the scripts are re-runnable or not, and afraid of further screwing up my configuration, I didn’t try that.

Instead, I figured I could figure out whatever steps the scripts missed and just manually do those.  So I built the sample, fired it up and got a home screen – so not all is bad, it could at least connect to the database under local dev fabric and display items.  But when I clicked “log in” the problems started.

The code apparently uses AppFabric Labs identity providers to allow for sign-in across a number of services – Windows Live, Google, Facebook, etc.  That’s great and one reason I picked it.  However, it does this by using some JavaScript to retrieve the identity providers as JSON and then populate the login screen – so debugging why that isn’t working isn’t too straightforwar.  The symptoms are that I just see a “Please Wait” screen when I click login; this is the default HTML for that division on the page, which is replaced by a JavaScript callback when the JSON for the identity providers is delivered.  So apparently it is never getting called back to repopulate the content of that division.

I figured I’d just try the URL the page issues to retrieve the AppFabric Labs identity providers and see what I got.  The URL is something like: (spacing added for readability):

https://bidnow-sample.accesscontrol.appfabriclabs.com:443/v2/metadata/IdentityProviders.js?
                 protocol=wsfederation&realm=http%3a%2f%2flocalhost%3a8080%2f&
                 reply_to=http%3a%2f%2flocalhost%3a8080%2fLogOn.aspx&version=1.0

It seemed wrong to me that it was referring to the bidnow-sample namespace within AppFabric labs, not the namespace I had created, so I figured that was probably one of the things that didn’t get updated by the scripts.  However, plugging this URL into a web browser, I get this:

Hmm, so it’s not complaining about an invalid namespace, as I’d expect; it’s complaining about the realm (whatever the hell that is) and which appears to point to the Azure local dev fabric.  Looking a bit into what a “realm” is, I find that this is the URI that the issued token from ACS will be valid for.  Searching a bit on the web, I found this page which explains how to set up ACS with Windows Azure.  It explains that when registering the “relying party application” with ACS, you have to specify the URI – I didn’t do that when I set up my AppFabrics lab info.  (This page also has more in-depth information about “relying party applications” and the realm – one of the challenging things about learning any new technology like this is that there are a bunch of new terms which you have to first learn; a good resource here is the December 2010 MSDN article “reintroducing” :) ACS; I guess you have to write an article re-introducing something when the first documentation on this just led to ho-hums and scratching heads.  But this article actually guides you through the necessary steps of configuring the ACS namespace and realm and all up on the AppFabrics Labs web site reasonably well.

Now the MSDN article is concerned with a generic ASP.Net web site, not the Azure BidNow sample, so there’s a bit of reading between the lines here.  After getting you to configure ACS, the MSDN article has you add an STS Reference to your web site.  BidNow already has this configured, so you just have to change the configuration of this.  Open the web.config file in the BidNow.Web project and change the places where it refers to bidnow-sample.accesscontrol… to <namespace>.accesscontrol… where <namespace> is the name of your ACS namespace, created as the MSDN article describes.

You can do this manually, or you can just follow the steps in the article by clicking “Add STS Reference…” to the BidNow.Web project.  The directions in the article are a bit out of date – it seems that some of the UI has changed – but basically, you have to click through, paste in the information from the ACS web site as described (to the “WS-Federation Metadata” URL).  You also have to tell the wizard to generate a certificate for your application – the default of “no encryption” didn’t work for me.

This got me past the first step – when I build and run the app and go to the BidNow home page and click “Login”, I now get a list of providers:

So far, so good.  Clicking on Google takes me to the Google federated login and I am able to login using my GMail account – I am then prompted to share my GMail email address with the ACS – this is as expected and is covered in the article.  However, when I click Approve here, I get this error:

Hmm…  Searching for this error on the web, I find a helpful explanation on acs.codeplex.com:

The rule group(s) associated with the chosen relying party has no rules that are applicable to the claims generated by your identity provider. Configure some rules in a rule group associated with your relying party, or generate passthrough rules using the rule group editor.

OK, so where in the article it told me “just use default rules”, I guess it wasn’t quite right.  Actually, it was – but I missed a step in my haste.  I forgot to click the “Generate Rules” for the default rules group for the providers selected on the ACS portal:

Now you would think that default rules would be, oh, I don’t know, defaulted but apparently not unless you click the button to generate them.  Yup, I’m understanding more and more why they had to write an article re-introducing this service.

After generating these, going back to my dev fabric hosted BidNow, I am able to login!

As is often the case, I figured I must not be the first person to want to do this, so there’s probably info on the web about it.   Back when I worked as a dev on Office, we had a strategy of being a universal receiver – no matter what format you had, you could open it in Office.  This was in the days when WordPerfect 4.1 and Lotus were the market leaders, so it makes sense to make it super easy for people to move from what they have to your format.  I figured the same must be true for Hyper-V, the kind of underdog in the VM space.

Sure enough, I found a good blog post from John Robbins describing how he did this.  John is doing a more complex migration than I need –he’s moving his whole environment, including an Active Directory domain controller from VMWare to Hyper-V.  I just have a couple of virtual disk images I need to be able to run under Hyper-V.  But John’s post links to just what I needed - a tool which does a sector-by-sector conversion from the VMWare .VMDK format for virtual hard disks to Hyper-V’s VHD format. 

But that gives me a virtual hard drive with the image – it doesn’t give me a virtual machine.  Here are the steps to convert that from the VMWare virtual machine information provided:

1. Download the VMDK to VHD Converter from VMToolkit.
2. Use it to convert the VMWare VMDK (virtual disk image) to a Hyper-V VDK (virtual disk image).  This creates a new file that is a sector-by-sector copy of the original virtual hard disk.
3. Start Hyper-V Manager and click on your server name in the tree control on the left.
4. Click New / Virtual Machine… and name it and configure memory/networking.
5. When you get to step 4 (Connect Virtual Hard Disk), click the second option “Use an existing virtual hard disk” and point it at the VHD you created from the VMDK.
6. Start the Virtual Machine.    Depending on whether the virtual configuration is significantly different than the VMWare image you received, Windows may need to configure hardware and restart the VM – this will happen automatically.

That’s about it – pretty easy migration from VMWare to Hyper-V!

I then went to pivot the data by clicking “Summarize with Pivot Table” in the “Table Tools” ribbon section, but the pivot table field list doesn’t contain my group column.

At first I thought that maybe for some reason calculated fields wouldn’t be included in the pivot table – but this made no sense and there are are other calculated fields in my source data.

After poking around a bit on the web, I found this post which describes the problem and how to fix it.  Basically, Excel has a “pivot table” cache which needs to get refreshed.  Since other pivot tables had been created in the workbook based on my source data by the person who gave it to me, Excel “knew” what the source data looked like – and in its view, it didn’t have a “Group” column.  Simply going to the pivot table sheet, selecting the “Pivot Table Options” tab and clicking “Refresh” made the “Group” column appear in my pivot table field list.

Springsteen put it this way:

And I said man, there’s other guys that play guitar well. There's other guys that really front well. There’s other rocking bands out there. But the writing and the imagining of a world, that's a particular thing, you know, that's a single fingerprint. All the filmmakers we love, all the writers we love, all the songwriters we love, they have they put their fingerprint on your imagination and then on - in your heart and on your soul. That was something that I'd felt, you know, felt touched by. And I said well, I want to do that.

Fred Brooks (author of “The Design of Design”, and of course, famously, “The Mythical Man-Month”) put it a bit differently:

“Great design does not come from great processes; it comes from great designers.    Choose a chief designer separate from the manager and give him authority over the design and your trust.”

Both are really saying the same thing – great, consistent, beautiful designs always come from a single mind expressing himself or herself.

I had lunch with a friend who now works at Facebook and he told me he’s working alone on his project – if he gets to the point where he needs others, he will enlist them, but so far he hasn’t.  He said this is how most things happen at Facebook and the processes would make an experienced software manager like me aghast.  No specs.  No schedules.  No testing beyond developer-driven testing to a percentage of users on the live site.  The ability for any developer to deploy to the live site.  But it’s hard to argue with the results.  This may be how to scale out – have a consistent design (Zuckerberg still oversees the overall design of the site, according to David Kirkpatrick, author of the Facebook Effect) but then allow for autonomy within that design.

I went to the "Network and Sharing Center" to see if there was some setting I had to tweak and found this:

Hmmm... so I'm kinda/sorta still part of a homegroup, it sounds like.  Searching for the message highlighted I found this Windows Online help topic that explains it:

http://windows.microsoft.com/en-US/windows7/Switching-between-your-home-and-workplace-networks

So it turns out that yes, you can see other computers from the domain-joined laptop, but the other computers on the homegroup can't see the domain-joined laptop any longer.  I suspect this is a security thing.

I found this post which explained that it could be an old read receipt.  I downloaded the tool referenced in there and ran it but didn’t find any read receipts – but did find two old email messages sitting in outbox for a Personal Folders file (not the default PST as the post mentions – that one was empty, but there was another one that had messages sitting in it).  Deleting those as described in the post (admittedly a rather geeky process that involves using this tool to call specific MAPI APIs, but it’s well outlined in the blog post how to do this) fixed the problem.

There is a $50 tool called OutlookSpy (with a  free 30-day trial) that was also mentioned and might help some folks who find the tool I used a bit too geeky.

To gather the data, I use a timer which I put into a simple value class:

[Serializable]
public struct TimerInstance
    {
        public TimerInstance(string str, Int32 nTicks)
        {
            _name = str;
            _ticks = nTicks;
        }

        private readonly string _name;
        private readonly Int32 _ticks;

        public string Name { get { return _name == null ? String.Empty : _name; } }
        public Int32 Ticks { get { return _ticks; } }

        public override string ToString()
        {
            return string.Format("{0,20}: {1,10:N}", Name, Ticks);
        }
    }

Then I just have a list of these which I add to as I run each test:

    static private List<TimerInstance> _Timers = new List<TimerInstance>();

At the end of a batch of tests, I want to persist out all the timer values into XML. I decided it was easiest to just store those in Azure blob storage. I can then import these into Excel where I can do some analysis to figure out the average overhead, minimum, etc.

This seemed super easy, but I found that serializing to XML is not as straightforward as I thought it would be.

First, I added the <pre>[Serializable]</pre> attribute to the struct (as shown above).  But what I ended up with was something like this:

<?xml version="1.0"?>

<ArrayOfTimerInstance xmlns:xsd="<a href="http://www.w3.org/2001/XMLSchema"">
                                    http://www.w3.org/2001/XMLSchema"</a>
                      xmlns:xsi="<a href="http://www.w3.org/2001/XMLSchema-instance"">
                                    http://www.w3.org/2001/XMLSchema-instance"</a>>


    <TimerInstance />

    <TimerInstance />
    <TimerInstance />
    <TimerInstance />

</ArrayOfTimerInstance>

I couldn’t figure out why the public properties (Name and Ticks) of the items in the list weren’t getting serialized.

After poking around a bit, I realized two things:

• The Serialize attribute has nothing to do, really, with XML Serialization – it marks the object as binary serializable which I wasn’t interested in for this.
• There is an interface, IXmlSerializable, you can implement on a struct or a class to control how XML serialization happens.

So I modified my struct as follows:

    public struct TimerInstance : IXmlSerializable
    {
        public TimerInstance(string str, Int32 nTicks)
        {
            _name = str;
            _ticks = nTicks;
        }

        private readonly string _name;
        private readonly Int32 _ticks;

        public string Name { get { return _name == null ? String.Empty : _name; } }
        public Int32 Ticks { get { return _ticks; } }

        public override string ToString()
        {
            return string.Format("{0,20}: {1,10:N}", Name, Ticks);
        }

        #region IXMLSerializable Members

        public System.Xml.Schema.XmlSchema GetSchema()
        {
            throw new ApplicationException("This method is not implemented");
        }

        public void ReadXml(System.Xml.XmlReader reader)
        {
            throw new ApplicationException("XML deserialization of TimerInstance not supported");
        }

        public void WriteXml(System.Xml.XmlWriter writer)
        {
            writer.WriteStartAttribute("Name");
            writer.WriteString(Name);
            writer.WriteEndAttribute();

            writer.WriteStartAttribute("Ticks");
            writer.WriteString(Ticks.ToString());
            writer.WriteEndAttribute();
        }

        #endregion
    }

and used this code to serialize it into a MemoryStream:

        // Return items that should be persisted.  By convention, we are eliminating the "outlier"
        // values which I've defined as the top and bottom 5% of timer values.
        private static IEnumerable ItemsToPersist()
        {
            // Eliminate top and bottom 5% of timers from the enumeration.  Figure out how many items
            // to skip on both ends.  Assuming more than 10 timers in the list, otherwise just take
            // all of them
            int nTimers = (_Timers.Count > 10 ? _Timers.Count : 0);
            int iFivePercentOfTimers = nTimers / 20;
            int iNinetyPercentOfTimers = _Timers.Count - iFivePercentOfTimers * 2;

            return (from x in _Timers
                    orderby x.Ticks descending
                    select x).Skip(iFivePercentOfTimers).Take(iNinetyPercentOfTimers);
        }

        // Serialize the timer list as XML to a stream - for storing in an Azure Blob
        public static void SerializeTimersToStream(Stream s)
        {
            if (_Timers.Count > 0)
            {
                System.Xml.Serialization.XmlSerializer x = new System.Xml.Serialization.XmlSerializer(typeof(List));
                x.Serialize(s, ItemsToPersist().ToList());
             }
         }

Notice that I have a LINQ enumerator which sorts the list and eliminates the outliers - which I define as the bottom and top 5% of timer values. Probably something wierd was going on when these happened, so I don't want the outliers to skew the mean of my overhead calculation.

Once I have this, it's a pretty straightforward thing to store the memory stream in an Azure Blob using UploadFromStream:

        // Serialize the timers into XML and store in a blob in Azure
        private void SerializeTimersToBlob(string strTimerGroupName)
        {
            // add the binary to blob storage
            try
            {
                // Create container if it doesn't exist.  Make it publicly accessible.
                if (_blobContainer == null)
                {
                    CloudBlobClient storage = _storageAccount.CreateCloudBlobClient();
                    storage.GetContainerReference("timers").CreateIfNotExist();
                    if ((_blobContainer = storage.GetContainerReference("timers")) == null)
                        throw new ApplicationException("Failed to create Azure Blob Container for timers");
                    _blobContainer.SetPermissions(
                                      new BlobContainerPermissions()
                                                 { PublicAccess = BlobContainerPublicAccessType.Blob });
                }

                // Remove invalid characters from timer group name and add XML extension to create Blob name.
                string  strBlobName = strTimerGroupName.Replace(':', '.') + ".xml";
                var blob = _blobContainer.GetBlobReference(strBlobName);
                MemoryStream ms = new MemoryStream();

                // Serialize all the recorded timers in XML into the stream
                TraceTimer.SerializeTimersToStream(ms);

                // Seek back to the beginning of the stream.
                ms.Seek(0, SeekOrigin.Begin);

                // Save the stream as the content of the Azure blob
                blob.Properties.ContentType = "text/xml";
                blob.UploadFromStream(ms);
                Diagnostics.WriteDiagnosticInfo(Diagnostics.ConfigTrace,
                                                TraceEventType.Information,
                                                WorkerDiagnostics.TraceEventID.traceFlow, "Added " + strBlobName);
            }
            catch (Exception exp)
            {
                Diagnostics.WriteExceptionInfo(Diagnostics.ConfigTrace,
                                               "Saving Timer Information to Azure Blob", exp);
            }
        }

(Diagnostics is my own class to provide TraceSource-based logging to Azure). The only additional trick I needed here was to Seek on the MemoryStream to the beginning - I was getting empty blobs before I did that.

The final result:

<?xml version="1.0"?>
<ArrayOfTimerInstance xmlns:xsi="<a href="http://www.w3.org/2001/XMLSchema-instance"">
                                    http://www.w3.org/2001/XMLSchema-instance"</a>
                      xmlns:xsd="<a href="http://www.w3.org/2001/XMLSchema"">
                                    http://www.w3.org/2001/XMLSchema"</a>>
  <TimerInstance Name="Fibonacci Generation (Trace) 1" Ticks="468294" />
  <TimerInstance Name="Fibonacci Generation (Trace) 9" Ticks="410877" />
  <TimerInstance Name="Fibonacci Generation (Trace) 3" Ticks="402439" />
  <TimerInstance Name="Fibonacci Generation (Trace) 5" Ticks="388574" />
  <TimerInstance Name="Fibonacci Generation (Trace) 8" Ticks="385690" />
  <TimerInstance Name="Fibonacci Generation (Trace) 2" Ticks="385261" />
  <TimerInstance Name="Fibonacci Generation (Trace) 10" Ticks="376425" />
  <TimerInstance Name="Fibonacci Generation (Trace) 6" Ticks="345973" />
  <TimerInstance Name="Fibonacci Generation (Trace) 7" Ticks="339461" />
  <TimerInstance Name="Fibonacci Generation (Trace) 4" Ticks="331053" />
</ArrayOfTimerInstance>
 

The problem is that very large address spaces encourage very large “in-memory” data structures (“The user paid for 64 bits of address space,” Kamp writes, “and I am not afraid to use it”) but those structures are not really in (physical) memory.  Depending on the virtual memory “pressure” (i.e. how many logical VM pages are actually on disk) memory accesses can vary by a factor of ten (if a page has to be retrieved from disk, even a very fast disk obviously cannot match the performance of RAM).  So algorithms that pretend that all memory accesses are equal can be quite inefficient compared to the algorithm for the Varnish cache that Kamp describes which "actively exploits” the fact that it is running in a VM environment.

There is a bigger point here, which is that all the layers of abstraction we’ve built (virtual machines, IL, class libraries, etc.) can be quite seductive. I made an entire class serializable into XML yesterday by adding one line of code (“[Serializable]” to the class declaration).  But part of me thought, hmmm, I wonder what the compiler actually generates now to do that?  It’s important that especially in hotspots we are aware of what the cost of abstraction is and actively look for ways to reduce it. 

Many of the comments about this article on the ACM site for this article can be summed up as “ho hum – you think you’ve discovered something new?”  But others are more in agreement with my perspective: I have been a practicing computer scientist (i.e. I write and design computer software for a living) for thirty years and I have not seen something as clearly written as this article about how to write code to exploit a VM environment.  The fact that some can point to theoretical pieces that have been written just shows how removed from practice that theory is.  But it is also a sign of how much the CACM has changed that it is running articles like this interesting to practitioners.