https://technet.microsoft.com/en-us/magazine/hh127072.aspx

Microsoft Services has designed, built and implemented a Private Cloud/IaaS reference implementation using Windows Server, Hyper-V and System Center. Our goal throughout this four-part series will be to show how you can integrate and deploy each of the component products as a solution while providing the essential cloud attributes such as elasticity, resource pooling and self-service.

In this first article, we’ll define Private Cloud/IaaS, describe the cloud attributes and datacenter design principles used as requirements, then detail the reference architecture created to meet those requirements. In parts two and three, we’ll describe the detailed design of the reference architecture, each of the layers and products contained within, as well as the process and workflow automation. Finally, in part four we’ll describe the deployment automation created using the Microsoft Deployment Toolkit and Hydration Framework for consistent and repeatable implementations.

For more information about the architecture, see the Hyper-V Cloud Deployment Guides here (also co-authored by Adam and I) and the MCS Datacenter Services Solution here. Stay tuned for the remaining three articles...

My general requirements are:

• Portability (I never check bags so the rig must fit in a laptop bag plus carry-on)
• Ability to function without any Internet connectivity (never know what type of customer environment I’ll run into)
• Ability to support multiple primary scenarios (Server Virtualization/Private Cloud, VDI, etc.)
• Modularity: For more complicated scenarios, I need to be able to add machines without lots of rebuilding
• Also function as my primary CorpNet machine

So the solution that works for me is one really powerful laptop augmented as needed by additional laptops/netbooks and supporting hardware. My primary laptop is a Dell M6400 with Quad Core mobile processor, 16GB of RAM, an internal 240GB SSD, and an internal 320GB 7200RPM drive. It also has an E-SATA port for an external hard drive. I dual boot between Windows 7 and Windows Server 2008 R2 partitions on the SSD. The Windows 7 install is my day-to-day OS for CorpNet access, email, etc. This runs Bitlocker and everything else required for compliance with Microsoft policies. The Windows Server 2008 R2 partition is my demo environment and runs a number of roles and applications in the parent partition (Hyper-V, AD, SQL, VMM, OpsMgr, iSCSI Target). Even with all that running in the parent, I still have plenty of resources for running VMs.

My VMs fall into one of two categories: Integrated demo VMs where they are members of "my" AD domain in the parent partition or stand-alone VMs where they don't integrate with my parent partition and services. The two integrated scenarios I have are Server Virtualization / Private Cloud and VDI. Below is the setup for my VDI scenario. My demo laptop is augmented by a Netbook and a wireless access point. For VMs, I’m running the full stack required to show all of the joint Microsoft + Citrix VDI solution. With this setup I can show Server, Desktop, Application, and Presentation virtualization with full fidelity user experience including HD video over a wireless connection. I will soon be augmenting this with Dynamic Memory and RemoteFX when SP1 comes out.

The next scenario, Server Virtualization / Private Cloud, requires some more gear. Since I want to be able to show all of the Hyper-V and VMM capabilities like failover clustering, Live Migration, etc. I need to have a host cluster of at least two machines. So I add two more laptops to the demo rig. Ideally these would be the smallest laptops that support Hyper-V. In my case, they are two Dell D820s. Now that Hyper-V R2 supports dissimilar cluster nodes via processor compatibility mode, the two machines don’t have to be identical anymore, they just need to have the same processor vendor (i.e. all Intel or all AMD). To create a host cluster you need shared storage and the simplest way to do this is to put a software iSCSI target in the main demo laptop. If you have MSDN access you can get Windows Storage Server and the MS iSCSI target for this purpose or there are a number of 3rd party options, just remember that you will need one that supports connections from multiple servers since there will be two in the cluster. All three laptops get connected to a small Ethernet switch, and a Hyper-V host cluster is configured using the two additional laptops. The cluster is added to the VMM 2008 R2 console, added into Operations Manager, etc. I also have both the VMM Self-Service Portal 2.0 (SSPv2) release candidate as well as the Dynamic Datacenter Toolkit for Hosters (DDTK-H) installed to show a variety of self-service VM provisioning scenarios.

The diagram below illustrates what this looks like. The only downside to this setup is the weight, 3 machines plus accessories, and the funny looks you get from airport security when you’re at the end of the X-Ray machine and taking every laptop that comes out However, with this setup you can demo pretty much every single capability the Microsoft stack provides including Live Migration, Performance and Resource Optimization (PRO), Failover Clustering, etc.

As a preview of blog posts to come, note the inclusion of System Center Service Manager 2010 and Opalis in my demo rig. I’ve mentioned in my recent posts the work I’m doing with a bunch of other top folks in Microsoft Services around new offerings related to private cloud and infrastructure as a service (IaaS) using our current products. Using all of the products above, we’re doing some pretty amazing orchestration and automation of physical and virtual resources. For a conceptual overview of what this is all about, see my Architecture Journal paper. I’ll be detailing our efforts a lot more in the coming weeks and months.

I knew Virtual Machine Manager 2008 R2 can work in this type of scenario but not having looked at it in depth before, I had to do some digging to get around a few problems, hence this post.

First, the couple issues I ran into would have been avoided if I had read all the documentation first. Since none of us ever do that, it was Bing to the rescue.

So here are the three main things to remember when trying to add Hyper-V host clusters in one forest, to a Virtual Machine Manager 2008 R2 installation in a different forest:

• A Windows Server-based host can be in a domain separate from the VMM server's domain and a host can be in a domain with a two-way trust with the VMM server’s domain or in a domain that does not have a two-way trust with the VMM server’s domain.

This I knew and I configured a two-way forest trust between my two forests.

• VMM does not support managing a host cluster on a perimeter network or in an AD domain that does not have a two-way trust with the VMM server’s AD domain.

This I did not know, but I had configured correctly already.

• Before you add a host cluster that is in a disjointed namespace to a VMM server that is not in a disjointed namespace, you must add the DNS suffix for the host cluster to the TCP/IP connection settings on the VMM server.

This was the main issue I ran into. DNS was otherwise configured correctly, the trust verified, but when I went to add the hosts and/or host cluster to VMM, I got an error 404 “The requested name is valid, but no data of the required type was found”. It was able to search AD for the server names but failed trying to add them. Following the documentation above, adding the DNS suffix to VMM for the domain the hosts was in resolved the issue and allowed me to add the hosts.

The following documentation links provide all the info needed for this scenario:

https://technet.microsoft.com/en-us/library/cc764275.aspx

https://technet.microsoft.com/en-us/library/cc917879.aspx

https://technet.microsoft.com/en-us/library/ee236431.aspx

“Managing virtual switches should be done from a central point and the configuration should be automatically distributed to the host operating system in such a way as to eliminate the need for a separate configuration switches for each of the hosts.”

The requirement to “manage” is pretty generic, that could mean anything. Minimally I’d assume it means create/update/delete virtual switches. The System Center Virtual Machine Manager 2008 R2 GUI supports these operations but only on a host by host basis. Here’s what you can do per host:

So per host you can create, update, and delete virtual switches. Since this is VMM, anything you can do in the GUI you can do via PowerShell and extend. Scripts can be stored in the VMM library and executed from there:

Now when these run, you get presented with a PowerShell shell that you can interact with if needed or just have the script run without input. You can prompt for user input or get fancy with a menu system. There are multiple cmdlets that let you do things with virtual networks and switches like New-VirtualNetwork, Set-VirtualNetwork, Get-VirtualNetwork, Remove-VirtualNetwork. From the VMM PowerShell console, you can do a get-help on these or see the reference document linked below. 

So for anything you need to do in a distributed fashion across multiple hosts, like “add new vNetwork to all hosts, set it as External, and connect to the Broadcom NIC” you can create a script to do so then any list of hosts you pipe to that script would have the script run against it. If you feed it all hosts, then it would run on all hosts. Here’s an example:

Get-VMMServer -ComputerName "VMMServer1.Contoso.com"
$VMHost = Get-VMHost -ComputerName "VMHost01.Contoso.com"
$HostAdapter = Get-VMHostNetworkAdapter –VMMServer “VMMServer1.Contoso.com” | where { $_.Name -match "Broadcom" }
New-VirtualNetwork -Name "External" -VMHost $VMHost -VMHostNetworkAdapter $HostAdapter

This is the code for a single host, so would execute against VMHost01. The selection criteria for which physical NIC to bind the virtual network to can be any attribute of the network adapter object such as Name(the physical adapter name), MAC, IP, etc. There are several methods that could be used to run this against multiple servers. This code fragment could be turned into a function with parameters, it could be executed inside of a loop or pipeline etc. To execute against all hosts in a particular host group, you could do something like this:

$VMHostGroup = Get-VMHostGroup -Name "MyHostGroup" -VMMServer "VMMServer1.Contoso.com"
$VMHosts = Get-VMHost -VMHostGroup $VMHostGroup
ForEach ($VMHost in $VMHosts) {

    #Anything you want to run on all hosts in the $VMHostGroup

}

The above would work for a host group of stand-alone hosts. Where this gets a bit complicated is that hosts groups, host clusters, and hosts are arranged hierarchically and can be nested. So if you have a larger structure with multiple host groups, each containing multiple clusters, and so on, you would need to use a few additional techniques.

To get all stand-alone hosts in a host group with child host groups, you can use:

$VMHosts = $VMHostGroup.AllChildGroups | Get-VMHost 

To get the host clusters in a host group, you can use:

$VMHostClusters = $VMHostGroup.AllChildGroups | Get-VMHostCluster

To get the hosts in a host cluster, you can use:

$VMHosts = Get-VMHost -VMHostCluster $VMHostCluster

So to be sure that you get all hosts from a given level of the hierarchy down, you need something like this (since this starts at All Hosts, it should run against every host in the hierarchy):

$VMHostGroup = Get-VMHostGroup -Name "All Hosts" -VMMServer "VMMServer1.Contoso.com"
if ($VMHostGroup.AllChildGroups -ne $null)
{
     $VMHosts = $VMHostGroup.AllChildGroups | Get-VMHost
     $VMHostClusters = $VMHostGroup.AllChildGroups | Get-VMHostCluster
     ForEach ($VMHost in $VMHosts)
     {
        #Anything you want to run on all hosts in the $VMHostGroup and all its child host groups
     }
}
else
{
     $VMHosts = Get-VMHost -VMHostGroup $VMHostGroup
     $VMHostClusters = $VMHostGroup | Get-VMHostCluster
     ForEach ($VMHost in $VMHosts)
     {
        #Anything you want to run on all hosts in the $VMHostGroup
     }
}
if ($VMHostClusters -ne $null)
{   
     foreach ($VMHostCluster in $VMHostClusters)
     {
           $VMHosts = Get-VMHost -VMHostCluster $VMHostCluster
           ForEach ($VMHost in $VMHosts)
           {
                #Anything you want to run on all hosts in the $VMHostCluster
           }
     }
}

For details on what you can do with the VMM and Failover Cluster cmdlets, here’s some links:

VMM 2008 R2 Cmdlet Reference

https://www.microsoft.com/downloads/en/details.aspx?displaylang=en&FamilyID=08a9eeb9-86ff-4a4f-8dfa-165bd701aba9

Failover Cluster Cmdlet Reference

https://technet.microsoft.com/en-us/library/ee461009.aspx

If there’s something that can’t be done from VMM, then check out the Hyper-V PowerShell Library on CodePlex:

https://pshyperv.codeplex.com/releases/view/38769

https://blogs.msdn.com/b/rds/archive/2010/07/13/beta-testing-microsoft-remotefx-in-service-pack-1.aspx

https://blogs.msdn.com/b/virtual_pc_guy/archive/2010/07/13/dynamic-memory-resources.aspx

For RemoteFX, be sure to closely review the hardware considerations document to ensure your test systems meet the requirements and also note that the blog post calls out some particular nVidia and ATI driver versions known to provide a decent experience.

I’ve been evaluating both RemoteFX and Dynamic Memory for some upcoming training I’m delivering. In a very short amount of time you can learn to implement and configure. The settings that are exposed are explained well in the documentation. For a deeper understanding of what’s going on, testing in various configurations and loads is a good idea while looking at all the new performance counters that have been added. That will give a good look at what the system is doing in different scenarios.

Issue 24 of the Microsoft Architecture Journal focuses on virtualization. My paper, From Virtualization to Dynamic IT, has been published in this issue. My thesis:

Virtualization is a critical infrastructure-architecture layer that is required for achieving higher IT-maturity levels, but several others layers—such as automation, management, and orchestration—are equally important.

For the last several months I have been focused on virtualization and private cloud offerings. My paper outlines several architecture layers in addition to virtualization that are required for a robust cloud infrastructure. The different forms of virtualization including server, application, storage, and network virtualization are key foundational technologies but they must be complemented by automation, management, and orchestration layers in order to achieve the cloud attributes of elastic capacity, user-self service, and resource pooling. In the paper I define those layers, detail some of their key requirements, then discuss how together they form the foundation for cloud infrastructure as a service (IaaS), either public or private.

In keeping with the mission of the Journal, my paper focuses on architecture and doesn’t get into specific details of Microsoft product offerings. For that, stay tuned to this blog as I will be detailing in a series of blog posts some of the work myself and others are doing in this space using current Microsoft technologies such as Hyper-V, VMM, Opalis, PowerShell, and the rest of the System Center suite. We’re doing some quite interesting work around datacenter automation that is going to make its way into both Microsoft and partner offerings in the virtualization space.

To encourage this trend among our partners, Microsoft has created a Virtualization Practice Accelerator:

Designed to help you build a services practice around Microsoft solutions, Practice Accelerator is a comprehensive set of reusable tools, resources, and best practices introduced with training via Microsoft Live Meeting. Enabling you to successfully identify customer requirements and project scope, analyze customer needs, plan solution requirements, and deploy Microsoft Virtualization, Practice Accelerator can help you build your capacity. The complete documentation set includes project guides, templates, architecture guidance, and planning and design guides, as well as leave-behind materials for your customer.

Key Components of Practice Accelerator:

Practice Accelerator provides a detailed framework covering every stage, from pre-engagement to post delivery and includes:

• A series of reusable tools, templates, best practices, hands-on labs, and guides.

• Materials to help provide architecture design, as well as post-deployment maintenance and monitoring recommendations.

• Easily customizable reference architecture documents for you to design and provide to your customer.

• Project guidance, including templates for planning and managing your customers’ projects.

• Planning and design guides to help you provide pre-deployment recommendations.

One of the core components of the accelerator is content from the Server Virtualization with Advanced Management (SVAM) offering which I have worked on for several years now with a team of very skilled virtualization architects in Microsoft. SVAM is a service offering developed by Microsoft Services, available to partners, and focuses on several virtualization scenarios such as server consolidation, advanced management, VDI, etc. It includes some of the most detailed architecture, planning, and implementation guidance available for the Microsoft virtualization and management platforms.

There are two upcoming training sessions for the Virtualization Practice Accelerator:

• January 18–21, 2010

• February 1–4, 2010

Register Here

These questions and many more like them will be my focus this year and the primary subject matter of this blog. I have recently transitioned to a different role within Microsoft Services which will let me focus on these topics and related customer projects full time. Previously I have been a 100% customer facing architect in MCS covering infrastructure architecture with projects last year ranging across AD, SharePoint, OCS, Hyper-V, System Center, etc. While great in terms of breadth, it didn’t allow me to focus on the virtualization, management, and cloud computing topics as much as I would have liked.

In my new role I report to the CTO of Microsoft’s US Public Sector Services business as an Architect focusing on virtualization and dynamic datacenters. In this capacity I will be working even more closely with the relevant product and services groups, have greater involvement with the large opportunities and customer projects in this space, and provide leadership in expanding Microsoft’s virtualization and datacenter services and offerings.

While it took some time, Microsoft has finally turned the ship and is focused squarely on virtualization, dynamic datacenters, and cloud computing. Between Hyper-V R2, VMM R2, Azure, and BPOS plus acquisitions such as Calista, Kidaro (MED-V), and Opalis, it should be very clear that Microsoft is serious about this space.

The biggest benefit of the cloud computing hype cycle that we are in today is that it is forcing organizations to finally look at infrastructure architecture in a holistic way and ask fundamental questions like what services should be run on premise vs. off premise, how to implement a service provider/chargeback model for IT, how to develop “cloud ready” applications which can be deployed in private or public clouds and spanning both when desired?

These topics and many others should keep us a bit busy this year!

People’s opinions on this topic seem to correlate most with whether they believe the defining attributes of cloud computing are financial (only paying for capacity utilized, no capex, etc) or whether they believe the defining attributes are technical (shared infrastructure, scale-out architectures, dynamic provisioning). The folks who focus on the financial side tend to believe either there cannot be private clouds because all costs are still incurred by the organization or that there is no way a single organization will be able to drive costs as low as the large cloud providers can with economies of scale.

The folks who see cloud computing as more of an architecture pattern for applications and an infrastructure/operational model tend to believe that the approach is just as relevant for a public cloud provider as it is for a large internal IT organization.

I am firmly in the camp of those who believe private clouds are going to be an important part of IT for at least the next decade. I come to this view using my definition of a cloud which is: an infrastructure architecture, application development model, and operations management discipline that dynamically provide necessary  services whenever and wherever they are needed while sharing costs between all users.  

Using that definition, all manner of clouds including public, private, hybrid, etc. will exist. Will there be certain economies of scale that a Microsoft or Amazon with hundreds of thousands of servers will be able to achieve that a single business won’t? Of course. But there will also be a degree of customization and agility that private clouds will be able to achieve that large providers won’t.

The reason I believe that private clouds as a concept are something new is that this is the first time that all of IT (infrastructure, development, and operations) are being looked at holistically. This is much more than just being able to sling VMs about the datacenter. This about providing a cost effective infrastructure where code that addresses user needs, be it an app, a VM, or a service can be developed rapidly by using foundational services, deployed near real-time, scale as needed, and then be retired at the end of its useful life.

I’m excited by Microsoft’s opportunities along the full spectrum of the cloud. Azure is a very forward looking vision of the public cloud that I still don’t think most people are grasping yet. Likewise, Microsoft’s traditional on premise solutions are evolving very quickly toward both private cloud and public cloud implementations. To me the most important question that will determine our long term success is how well we are able to provide a seamless continuum between the Azure platform and our Server and Tools solutions as they evolve toward cloud services. I think we are targeting an end game that no one else is really going after from on-premise, through private cloud, to public cloud.