Orchestration across multiple data centers 

Recently I did some research for a customer on possible ways to distribute System Center Orchestrator 2012 over a wide area network,. The customer in this case has global data centers at several sites, but also needs some form of command and control at the enterprise level.  Some information I’ll present here was also contributed by some really smart guys in Microsoft’s Orchestrator product team – Jim Britt and Charles Joy.

In a scenario of orchestration and a wide area network there are at least 3 possible models:

1. Centralized - A centralized instance of Orchestrator automates IT administration in several remote sites
2. Decentralized – several instances of Orchestrator, where a localized instance of Orchestrator automates IT administration at each local site
3. Centralized “Stretch” Instance – a Centralized instance where the Runbook Server role is stretched out to remote data centers 

Below I’ll explain each model and describe the advantages / disadvantages of each model, and some things to be aware of for each.

What makes an Orchestrator instance ?

To set the terminology straight, what do we mean by an Orchestrator INSTANCE ?  Basically an instance is comprised of the four major server roles that Orchestrator requires:

• SQL Server Database – often provisioned as a dedicated instance on an existing SQL Server
• Management Server – the management server is the core of Orchestrator authoring, the Runbook Designer and Runbook Tester connect here.
• Runbook Server – Orchestrator explicitly supports having more than 1 Runbook Server per instance
• Web Services – this includes the Orchestrator Web Console and Orchestrator Web Services. These are normally deployed as a single unit to one or more servers.

So each instance of Orchestrator will look something like this:

Note there are all sorts of scalability and high availability models for Orchestrator, and those things have probably been well covered elsewhere. I will refer you to the Scale Planning information at https://technet.microsoft.com/en-us/library/hh420359.aspx.

1) The Centralized Model

Here we have a single instance of Orchestrator performing automation at several datacenter sites.

Advantages

• A single instance is easiest to manage overall
• Moving solutions from Development into Production will be straight forward
• It will be easier here to gain a consolidated view of automation results and/or issues across the enterprise

Disadvantages

• Network bandwidth (especially, high latency) may inhibit effectiveness of your runbook automations
• Firewalls between datacenters are likely to present a challenge 
• With a single instance there is more security planning and administration needed, to make sure teams or persons cannot overwrite each other’s projects. There may also be a risk concerning sensitive information that could be stored in Orchestrator (such as passwords).

2) The Decentralized Model

Here we have multiple instances of Orchestrator, perhaps one per datacenter. We could even have several instances per datacenter, depending on how various teams work together. The advantages and disadvantages of this approach will be pretty much the opposite of the centralized model.

Advantages

• Multiple instances provide easy segregation of duties and delegation of ownership / responsibility.
• Autonomy – if a new project team wants to start up their own instance of Orchestrator it doesn’t require an exception. the model.
• Runbook servers can be placed closest to the IT servers and fabric being managed.

Disadvantages

• Runbook synchronization and version control across instances is a manual process in this release. There are no supported automation interfaces for this sort of thing, and no over-arching management across multiple SCOrch instances.
• If you need to consolidate runbook status or progress across these 2 or more distributed instances, this will require custom work.
• More instances overall being managed, and these must be patched occasionally, monitored and kept in a healthy state.

3) The “Stretch Armstrong” Model

Here we have a single instance of Orchestrator , but with Runbook Servers from that instance deployed to the remote datacenter sites. This model is TO BE AVOIDED  in most cases, unless you have a highly reliable and speedy network (megabit or above) network connecting your WAN sites.

Advantages

• Ability to manage a single instance of Orchestrator
• Localized presence of Runbook Server on the same network as the managed servers and/or fabric

Disadvantages

• Bad Performance - This is a potential SHOW STOPPER if you have a slow WAN in place, because the Runbook Server (RS) will make frequent calls to the database server (DB).
• It may be more difficult to troubleshoot runbooks – because runbook logging from the runbook server to the Database will require additional bandwidth and may be disabled 
• It is actually complicated to control precisely which Runbook Server a Runbook will execute on. There is a setting of a Preferred Runbook Server for each runbook. But if you want a single runbook to automatically execute at the most optimal location this is not directly supported today.

The overall impact is you are likely to have bad performance,  and you may not achieve the gain in automation speed and reliability you are seeking.

Design principles for the Orchestrator WAN scenario

There are a few design principles we can highlight here:

• There is no overarching management console for Orchestrator to provide a consolidated view of multiple instances. In this release each Orchestrator instance is pretty much a stand-alone entity.
• If you have a Runbook Server on a different network from the Database Server, ensure this network connection is high speed and has a packet latency of no more that 15-20 milliseconds.  Also be aware if your network connection has a high error rate as this can slow down communications.
• When distributing copies of a runbook to multiple instances, be sure to approach this with centralized authoring in mind. You don’t want to have a set of runbooks being modified by several people in different places.
• If you need to invoke one Orchestrator instance from a separate instance, use the REST web services. This will be a stateless connection and will traverse a WAN relatively well.  Also there is a Powershell module in Codeplex to allow invoking a runbook via the Web Service from within Powershell. That module is available at https://orchestrator.codeplex.com/releases/view/82959.

References

A few additional references to point you in the right direction:

• Jim Britt’s article about similar work done with Opalis - https://blogs.technet.com/b/jbritt/archive/2011/05/06/evaluating-the-quot-reach-quot-of-our-opalis-infrastructure-at-microsoft.aspx.
• The Orchestrator product team blog - https://blogs.technet.com/b/orchestrator/ 
• The Orchestrator homeworld on Technet – https://technet.microsoft.com/en-us/library/hh237242.aspx
• The Orchestrator SDK on MSDN – msdn.microsoft.com – Toolkit and Web Services

I hope this information is helpful for your System Center Orchestrator endeavors.

Glenn Walton, July 2012

Available in print or E-Book, I really like the Sapien Powershell "TFM" book. I keep a link right on my desktop and pop it open frequently when i need a little help or inspiration.

One other great reference and tool is the Powershell Survival Guide.

Oh, and don't get too comfortable... Powershell V3 is coming. This will bring some amazing new capabilities to scripting on the Windows platform. See these links for New Features, and Articles. 

V

The input parameters for my runbook are: User common name, First Name, Last Name, and Group Name.

This is a good time to use some Orchestrator variables to store common data like : DomainDNSName, NewUsersOU, etc. 

The AD Integration Pack is a new feature for Orchestrator and has a base set of activities to enable managing AD objects, mostly limited to Computers, Users, and Groups.  The activities are shown here:

Install and configure the Integration Pack 

you can download the IP (Release Candidate) by following the link shown above. After using the Deployment Manager to deploy the IP you’ll need to edit the settings for your AD Integration Pack in Runbook Designer:

Build the Runbook

My test runbook is shown graphically in the figure below. A short description of each step:

• Initialize Data – takes the input params as mentioned above
• Generate Random – using the Generate Random Text activity (this can include lowercase, uppercase, numbers and symbols)
• Create User – minimally you must include the CommonName, Password, and Parent Distinguished Name (OU location for new user)
• [if we get an error from Create User we call an error handler and quit]
• Get Group – I found this one tricky at first. You have to enter the Properties which control the LDAP search, and one or more Filters to describe the Group object being searched for. I used the Common Name for my group filter.
• (Now at this point the tricky part. I found that Get Group returns a Success value even if it doesn’t find anything. So my subsequent Link criteria used “Distinguished Name does not contain CN=” if the group was not found. This took some trial and error to find the workable solution. It would be nice if Get Group returned a “Number of Objects Found” value to the databus but apparently it does not.
• If group already exists, I add the user to the Group
• If the group does not exist, create the Group and then add the users
• Enable the user account (because AD creates user accounts in disabled state by default)

The runbook looks like this:

Additional details on the Get Group activity:

Determine if Get Group found nothing:

Summary

The AD Integration Pack provides some easy to use automation steps which make a nice addition to Orchestrator. With a modicum of knowledge you can quickly put it to use in your workflows for onboarding  / offboarding / provisioning etc.

The runbook is attached to this posting. I did not export the Variables but they are shown here:

AD IP Test.ois_export

Anyway, in case you're wondering how long it takes to fully evacuate a Windows Server 2008 R2 Hyper-V node using SCVMM 2012 and a 'Start Maintenance Mode' operation, the answer of course is "IT DEPENDS".  For the systems i'm working with at this time i'm getting a run time of about 46 minutes to evacuate 20 highly available VMs from a Hyper-V node. VM migrations are processed one at a time and each VM is currently taking around 2 minutes to evacuate. One caveat with this number is that a heavily loaded VM running a high performance workload can take much longer to migrate.  The underlying technology which makes this "evacuation" possible is Hyper-V Live Migration. More information about Live Migration is available with this Guide and this FAQ.

The configuration is running on a set of new issue Dell blades with lots of CPU and RAM, high speed networking & storage.  For any serious data center management tasks this setup is fairly standard these days. 

If you need some advice and guidance on selecting hardware, see Microsoft's Hyper-V Fast Track program: https://www.microsoft.com/en-us/server-cloud/private-cloud/hyperv-cloud-fast-track.aspx.  The Fast Track program is being updated with new information for System Center 2012, and soon Windows Server 8 comes into sharper focus, so this program will be a good thing to keep an eye on.

With Maintenance Mode provided by VMM we can do various maintenance tasks on the cluster nodes, such as software updates, without incurring any downtime of the running virtual machines.  The Dynamic Datacenter is very exciting and it's happening now!

Glenn Walton

Infrastructure Architect, Microsoft Enterprise Services

•  Overview of a Service
• Creating Service Templates
• Technet – Creating and Deploying Services in VMM

This service can be a single-tier or a multi-tier application. In the case of a multi-instance web tier the service template feature can also update a load balancer automatically ( as I wrote about in my previous posting “Fabric Management in SCVMM 2012”. ) 

When working with Service Templates in a scripting / automation capacity its important to understand the object model that is exposed by VMM. There are 3 steps you will walk through in order to create and deploy a service:

1. Create the service template – this is a model of your service and you will want to use the graphical service template designer to do this
2. Create a service deployment configuration – this represents your service template translated to a real service, except it isn’t running yet. This is the deployment plan for the service, and you also save this in the VMM Library and deploy later. 
3. Deploy a service – this creates the running service composed of 1 or more tiers with 1 or more VMs. 

See Figure 1 below for the diagram of this, showing the relationships.

Now from a Powershell standpoint, you have some new objects you’ll need to get familiar with.

• SCService – represents a running service
• SCServiceConfiguration – represents a configured Service not yet deployed
• SCVirtualMachine – represents a running VM (either as part of a service or standalone. If you did any scripting with SCVMM 2008 you’re already familiar with this.
• SCVMConfiguration – represents a configured VM not yet deployed
• ServiceTemplate  - represents your Service as designed in the Service Template Designer

Each of these Powershell objects correspond directly to the configuration items shown in the diagram here; this is from the posting (also mentioned above) Creating Service Templates, by JC Hornbeck. 

Architecture best practices for automation

Before we jump into deeper level discussion of these automation steps, its important to set the architecture context for all this.

Automated service deployments should be traceable and should map to business services. This gets us into the notion of IT Service Management and ITIL’s “good practices”, and all the wonderful issues around people and process.  Here are some scripting examples using the concepts and structures we just discussed. This could be a long and involved discussion so I’ll take a few large shortcuts to show a conceptual / logical view of how this automation fits into the overall technology solutions context.

See also the Reference Architecture as explained in Cloud Computing: Architecting a Microsoft Private Cloud, which is based on the well-known NIST cloud definitions. 

Powershell CommandLets for Services

The commands – Powershell cmdlets - available will depend on which sort of object you’re working with. When you’re working with just the templates and the configuration objects you have a base set of cmdlets to Create/Read/Update/Delete the objects. Once you deploy a service you also have operational cmdlets like Start/Stop/Suspend/Resume:

Scripting Examples

Here are some scripting examples using the concepts and structures we just discussed.

Create a service configuration from a Service Template:

Rename the VMs in the Service Configuration:

## get the web tier object
$tierConfig = Get-SCComputerTierConfiguration ‘
-ServiceConfiguration $serviceConfig  ‘
| where { $_.name –like “Web*” }

## get the VMConfigurations within the tier
$vmConfig = Get-SCVMConfiguration -ComputerTierConfiguration $tierConfig

## Set a naming convention so VMs will be named like
## "MyVMServer001”, “MyVMServer002” etc.
$namePattern = “MyVMServer”

## get number of VMs already deployed w/ this name pattern
$vmList = Get-SCVirtualMachine -cloud $cloud | where-object { $_.Name –like ($namePattern + *) }
$seed = $vmList.count + 1

## set vm names in this tier according to the pattern
$z = 0
do
{
$vm_name = $namePattern + ($z + $seed)
$computer_name = $vm_name 
Set-SCVMConfiguration -VMConfiguration $VMConfig[$Z]  -name $vm_name  -computerName $computer_name
$z += 1
}
until ($z -ge $vmConfig.count) 

Deploy the service configuration to a running service:

##

## Update the Service Config
Update-SCServiceConfiguration -ServiceConfiguration $serviceconfig
## Deploy the service 
## note include –RunAsynchronously to
$newSvc = New-SCService -ServiceConfiguration $serviceconfig

## check deployment state and status
write-host “Deployment state: “ + $newSvc.deploymentState
write-host “Service status: “ + $newSvc.overallStatus 

Closing thoughts 

Within SCVMM it is very educational to watch the Jobs window as your scripts are running, this allows you to see how the script commands are carried out within SCVMM.  Also SCVMM always exposes the “Powershell" button so you can hop in to PS and try things out.

the following Get-Help topics will be very helpful, especially as of now when the formal documentation of the VMM cmdlets have not yet been released.

• about_VMM_2012
• about_VMM_2012_Cmdlet_and_Parameter_Name_Mapping
• about_VMM_2012_Cmdlet_Backward_Compatibility
• about_VMM_2012_Connecting_to_the_VMM_Server
• about_VMM_2012_Host_Platforms
• about_VMM_2012_Library
• about_VMM_2012_Role_Based_Security
• about_VMM_2012_Run_As_Accounts
• about_VMM_2012_Virtual_Networking
  

Also note, if you’re using Powershell Remoting you’ll need to begin your script with:

import-module -name "virtualmachinemanager"

We are entering into a tremendously exciting evolution of computing, where datacenter automation is essential. And through the capabilities in SCVMM 2012 (and other new-generation toolsets) this automation is something that can be achieved pretty easily.  The virtual machine metaphor allows us to treat servers like software objects, which they are.  With the power of SCVMM 2012 and others in the System Center 2012 wave, we can build solid reliable solutions for the automation, management and orchestration of the virtual machine layer.  This set of capabilities will allow for business applications to be deployed and managed in a dynamic and cost-effective manner. 

.

Glenn Walton | Senior Infrastructure Architect, Microsoft Services

The steps I’m describing here were done with the SCVMM 2012 Release Candidate. SCVMM is bringing about one of the big changes in automated datacenter management which is this notion of managing the compute/network/storage fabric within the datacenter. With SCVMM you can fully automate Hyper-V host deployments, storage and network setup and other elements. Along with this, keeping with the Microsoft mantra that Its All About the Application!, we can define service templates which can fully automate service deployment and application management.  There are so many great innovations in the System Center 2012 wave that I can’t stop thinking about the possibilities ! 

Load Balancer setup

The first part of load balancer setup was fairly easy, I downloaded the F5 provider (see References below), and installed it. Setting up the load balancer in VMM is straight forward, it includes the IP address of the device, a RunAs account (provide the credentials to connect and manage the device), selecting the Host Groups and Logical Networks that can access the device, etc.

At the final page of the setup you can test the load balancer connection from SCVMM.

Logical Networks and IP Address Pools

The other key to configuring your SCVMM environment is to set up IP address pools on one or more logical networks. In order to use a load balancer in a service template the VMs must have static TCPIP addressing. But rather than hard-coding addresses, it means the VMs will pull their addresses from a pool of addresses maintained on SCVMM. Think of this as a mini-DHCP environment only for VMs. 

Your IP Address Pool consists of:

• a network site with CIDR notation, I used 10.6.14.0/23.  This notation will be familiar to those of you who have set up Active Directory subnet boundaries. Along with the network site you associate this site to one or more VMM Host Groups.
• IP Address ranges and VIPs – VIPs are Virtual IPs which will be assigned to a service when you use the load balancer in a service template. More on that directly below.
• Gateway, DNS, and WINS info, just like setting up a DHCP scope.

Virtual IP (VIP) Templates

Then you’ll need a VIP Template. This consists of:

• Template Type – usually will be specific to a particular load balancer configuration
• Protocol – including HTTPS Pass-through or Terminate. These will be important if the application being deployed uses SSL.
• Persistence – if the application uses session state the load balancer will be able to keep a particular client connected to a particular server (VM) in the farm.
• Load balancing method
• Health monitors – this is the method by which the load balancer can detect the state of a member of the server farm and determine if its healthy. These monitors can be simple or more complex depending on what constitutes healthy for the environment and application. 

The Service Template

the template was simple in this case, just a 1-tier app layer with multiple web servers. In this case we had already deployed the SQL Server database for the application so it was not defined as part of the service.  SCVMM and SQL Server 2008 R2 can be combined to create fully automated database deployments, but we haven’t implemented that yet.

In the Service template designer we have this. The non-intuitive part is you have to draw the connections from the  load balancer Server connection to the NIC, and from the Client connection to the logical network. Note the IP address in configured as Static, this is important.

When you’re ready to Configure Deployment, SCVMM will map your service template onto the fabric and allow you to deploy the entire service in a fully automated fashion. This entire process can be Powershell-scripted of course, and in this case the requirement was to monitor for a Change Request in Service Manager, and then execute Powershell script from within an Opalis workflow. 

(In the figure below the Red X’s indicate some issue with VM placement, in actual practice these can be easily fixed). 

.

The Proof is in the Pudding

After deploying the service template I was able to go into the load balancer UI (using the F5 web admin tool) and see the configuration items that SCVMM created. Undoubtedly one can customize the Powershell statements to enforce some different naming conventions, or other changes.

One other note, F5 Networks also has a Powershell provider so you can manage your load balancer directly from Powershell. SCVMM presents a higher-level view of the network fabric. But if you need this lower level of control you can find the provider and sample code at https://devcentral.f5.com.  

Resources

Some resources you can peruse to learn more about fabric management:

SCVMM on Technet - https://technet.microsoft.com/en-us/library/gg610610.aspx.

SCVMM Technet - “How to Configure a Hardware Load Balancer for a Service Tier” and “How to add hardware load balancers”

SCVMM blog posting - Integrating Load balancers into SCVMM 2012 Beta

F5 Networks provider for SCVMM 2012 -  LINK; (note, you must sign in on F5’s web site to get the download).

The Microsoft Private Cloud blog - https://blogs.technet.com/b/privatecloud/ 

I love being a hands-on architect.

Glenn Walton | Senior Infrastructure Architect, Microsoft Services

You can find it here:
https://www.microsoft.com/downloads/en/details.aspx?FamilyID=a90c3145-19fe-46fb-baec-c333629f2630&displaylang=en

Although the health model is fairly simple (see figure below), the MP also has a lot of intelligence in monitors that know about many of the interesting events a FIM service might throw.

Other related links:

• Working with SCOM Management Packs
• Forefront Identity Manager operations guide
• SQL Server 2008 Operations

I hope you will find this information helpful.

These days we’re hearing so much about Cloud and (while there’s plenty of noise going on ) a lot of the discussion is about a very significant shift in how IT services are delivered.  So I’d like to start a series of posts I’ll call “Private Cloud Foundations” to talk about the architectural underpinnings of a private cloud.

Service Catalog management has been a well recognized topic in IT operations circles, and in some form or other a Service Catalog has been part of the ITIL best practices material since V1.  ITIL defines Service Catalog as “ [Service Catalogue ]  provides a central source of information on the IT
services delivered to the business by the service provider organization, ensuring that business areas can view an accurate, consistent picture of the IT services available, their details and status.”

Catalog Relationship to Private Cloud

In a private cloud we have a goal of lowering costs and increasing agility, with the ability to create new Services and deploy existing Services quickly and easily. We do these things by enabling many of the NIST cloud attributes, things like Automation and Elasticity. So it would seem obvious we need a standardized way of describing our services and interacting with them for improved service management.  The interesting thing though is that while a service catalog makes great sense and we can understand what it means, a lot of organizations still lack a well defined approach to building that Catalog and making it intelligent enough.  To build an intelligent service catalog will require several things. It should be:

• well organized and map to business scenarios
• based on a  reference model
• allow IT services to plug in at multiple levels. see the Taxonomy discussion below.
• easily query-able and self-describing
• version controlled and able to evolve over time
• provide a highly available service to support automation interfaces 
• etc….

Taxonomy and Reference Model

When I think of Taxonomy I usually think of how the plant and animal kingdom has been classified on an 8-tier level, everything from Life to Species is represented. In the IT world we need to start by creating a similar taxonomy for IT services.  This would form a reference model for our ideal Service Catalog, and from that model we can create a reference architecture and one or more implementations.

The SOA reference model provides some guidance here:

We can see from the diagram above that while we have an agreed upon ITIL definition of what a service catalog is, we have little or no standards, specification, profiles, or protocols that guide in how to build such a catalog. But many vendors have done this work to varying degrees, which I’ll explore in the next posting of this series.

Lets start by talking about a simplistic data model for a service catalog.  In the world of IT we would propose a six layer data representation of IT services:

• Component – a reuseable unit of work and functionality (Input – Process – Output)
• Feature – a product building block which delivers a known set of functionality, perhaps by combining several components and knitting them together
• Role – a composition of features. In Windows Server 2008 we have many well defined server roles such as Domain Controller. 
• Workload – a composing of related roles which run together
• Solution – a set of integrated workloads that address a specific problem
• Product – a solution packaged as a product

(For the above breakdown, please see Application Patterns for Green IT by Dan Rogers and Ulrich Homann.)

Our challenge in designing private cloud is to design and create the service catalog with the right level of detail  so it can be useful but not overwhelmingly complicated.  In today’s private cloud approach we have something in between a Feature and a Role, known as an OS Image, which is commonly deployed in a somewhat automated fashion. That image then must be further customized to create an actual Virtual Machine or VM. That VM in turn must be combined with many other systems (whether virtual or physical) to form a Workload, and so on.  What is desirable is the ability to design and deploy an entire Workload or Solution into a private cloud environment.

Microsoft is taking a significant step forward in solving this problem with some of the features announced in System Center Virtual Machine Manager 2012.  A key goal of SCVMM 2012 is to allow the unit of deployment in  a private cloud to be the Workload or the Solution, or more commonly something we call an “application”, rather than just a generic set of operating system images. In order to be fully successful with this new approach we will need to consider how we can design, build  and maintain a Service Catalog for the enterprise. 

In an upcoming post I'll briefly review some existing vendor implementations of Service Catalog functionality.

So I had a chance to work on a few additional enhancements to the LDIF-Based Offline GALSync solution.

In the area of FIM file management one of the needs was to maintain an archive of each LDIF Export file which is created. So I decided it would be best to do the following:

1. Run the LDIF Export routine via WMI script (as usual)

2. Copy the resulting file to a folder C:\Archive

3. Run a PowerShell script to copy the file and embed the timestamp in the filename. So we copy it from “LDIF_Export.ldif” to include the timestamp “LDIF_Export_MMDDYY_HHMMSS.ldif”.

A Powershell routine which will do this (and I had to sweat it out as I don’t spend a great many hours each week writing PowerShell!) looks like this…

#filename: CopyFileWithTimestamp.ps1

param( [string] $input_filename)

function get_new_filename
{
#
# this is a comment
# note, we assume the file exists...otherwise this will bomb out
#
param( [string] $infile)
$file1 = get-item $infile
$file2 = $file1.name.replace($file1.extension,"")
$destfile = $file1.directoryname + "\" + $file2  +  "~" + $file1.lastwritetime.tostring("MMddyyyy_HHmmss") + $file1.extension
return $destfile
}   # end function

#
$dest_filename = get_new_filename( $input_filename )
$file1 = get-item $input_filename
$file1.copyto($dest_filename)
write-host "file copied"

Putting it all together

Now the execution control for the LDIF Export process can be managed from a single .CMD file, which will be invoked via the Windows Scheduler, such as:

cscript LDIF_Export.vbs
Set LDIFFILE=My_LDIF_Export.ldif
copy "..\madata\LDIF_MA\%LDIFFILE%" c:\archive
powershell set-executionpolicy Unrestricted
powershell -command ".\CopyFileWithTimestamp.ps1 'c:\archive\%LDIFFILE%' "

 
This little PowerShell script should come in handy for other purposes in the future. Also it gave my scripting skill set a nice little workout.

Reporting on Run History

Another area for improvement is to create some notification between the two organizations around the ongoing synchronization process. It will be pretty easy to get all the FIM Run History data programmatically (via WMI) and perhaps send the results via email to the system administrators on both sides.  FIM features, as inherited from ILM 2007 FP1, are pretty strong in the WMI Provider classes which give us more precise control over the operational aspects of a solution.

The MIIS_RunHistory interface is documented here: https://msdn.microsoft.com/en-us/library/ms697834(v=VS.85).aspx. MIIS_RunHistory gives us a collection of each item in the Run History stored on the server. The MIIS_RunHistory class defines the RunDetails method, which is an XML collection defined as a run_history element.

If you do a Bing search on "MIIS_Runhistory samples" you'll find quite a number of helpful samples. These will be easy to incorporate into a more robust solution. 

Improving the File Transfer process

Another area of the solution I've been investigating is the Windows Azure table storage and the SQL Azure functionality. I'll expand on this portion of the solution in my next posting, after I finish thoroughly digesting a recent issue of MSDN Magazine which has some articles about Data In the Cloud.

Until then, its Happy Holidays and Merry Christmas to all. Hallelujah!

For one of our enterprise customers we developed a solution for what I will call an Offline GAL Sync, meaning we are sharing Exchange address book information across multiple disconnected organizations. We were not able to use the standard Forefront Identity Manager (FIM) Galsync management agent because this requires a live connection to both source and target systems from a single FIM instance.

The capability this solution provides is to:

• Export all or selected address book information from the Source organization
• Import that information to one or more Target organizations
• A one-way sync process, note that the data at the Target is to be treated as read-only; some solutions may require 2-way synchronizations

The basic architecture is shown here:

Now while this is a workable solution there are a few potential problems:

• Sequencing of updates across multiple FIM instances – if a delta update is missed on the Target side it could affect processing of future updates. What’s might help a lot here is some type of runbook automation which can span across 2 or more organizations. With System Center Opalis this could be done, although the current product may hit some stress points in working across organizations. 
• Ensuring file transfer is working – this must use an Internet connection
• automating the end to end process – the FIM portions are relatively straight forward to automate on their own, but now we need to introduce an event driven trigger to kick off processing on the Target side.

Now lets talk about some improvements that could be made.

Runbook Automation

I would like to see this overall process surrounded by a runbook automation process. With a tool such as Opalis in place, we could watch for files to drop in a particular folder and then trigger the FIM “Delta Load” process on the target. The runbook automation could also handle error conditions, provide notifications, and many other useful tasks to make the overall process more manageable. Our revised architecture would look like the following:

Opalis provides a rich set of workflow objects for file handling as shown in the figure below (this is from the Opalis Workflow Object List document).

File transfer over the Internet – Windows Azure to the rescue !

Now how about that kludgy file transfer process?  While there are 3rd party products available which do this type of thing and provide scripting interfaces, that seems an inelegant solution. Since pretty much everything has to work over the Internet these days I believe secure file transfers will become a free built-in service (at some point!). As to how this might be designed I’ll get some feedback from my Azure expert friends and post an update soon. I would imagine this has already been built for some projects, perhaps using the Azure Blob Storage service as described in this Walkthrough.

Assuming we have this Azure-based file transfer as a generic capability, our runbook process would now look something like:

Opalis Notification Workflow built-in objects are shown here:

…..

I’m convinced that Azure cloud features will bring a wealth of great new capabilities to Microsoft’s server-side infrastructure products such as Forefront Identity Manager and Opalis. Every day we will be challenged to think outside the box about these new architectural models.