Some information

My introduction post on what is coming out on the Building Clouds TechNet Page!

The Automation Track that I am providing input to along with my good friend Charles Joy.

Something fun my team did with 4 Surface Pros (a must watch)!

By the way the entire team is going to be at MMS this year so come by our booth if you are at MMS 2013.  Look for posts on the Building Clouds Blog for sessions presented by the team in April.

I’ll continue to post other items out here probably but not as frequent as the Building Clouds site.  Actually, I’m looking forward to potentially posting a Windows 8 Theme Pack of my pictures   www.jimbrittphotography.com (shameless plug) in the near future!

Until next time, Happy Automating!

Hello again readers!  I wanted to take a brief tangent from my technical content to let you all know about a fantastic book that Microsoft has been publishing for the last four years.  This book is an amazing representation of the internal talented photographers at Microsoft and even better, the proceeds to go the United Way to support a great cause!

OK so give me the details right?

• Price is $50 which gets you a 300+ page book with 453 images!
• $37.50 of that goes to charity. 
• AND if you are a US-Based Microsoft employee you are matched by Microsoft so a $50 purchase results in $75 going to charity!

Call to Action (GO GET YOUR BOOKS!)

I encourage you to check out the details of this book here: https://www.photographersatmicrosoft.com/. 

It goes on sale starting October 1st, 2012.  This book is truly beautiful. 

Details below from the site:

Shameless Plug

For the second year in a row since joining Microsoft, I’ve been lucky enough to be featured along with the serious talent represented in this amazing book.

This year’s photo

Last year’s photo

Check out the United Way stats!  What an amazing organization! 

I’ve seen a topic come up from time to time that I think was worth getting out to those that can use it .  This is already published on TechNet (referenced within my blog here) but I wanted to put some finer notes on it to assist.

Issue –  Updating the Primary Administrator and Orchestrator User Group Post Install

So consider the scenario where you have installed and setup Orchestrator, selecting some initial defaults that you were comfortable with for access to the designer and who is the primary administrator for Orchestrator,  then you want to update your environment to support a new group or primary administrator.  In order to change the primary group and / or primary administrator after the fact, you need to leverage a tool called PermissionsConfig.exe located on the Management server.  You can follow the below process to accomplish this.

Changing User / Group Rights After Install

The following process is documented as a supported method to change the admin user and administrator group that is used by Orchestrator.  On occasion, you may need to update one of these (example: picked the wrong group during installation, user that setup Orchestrator is not going to be supporting it going forward).

• Go to an Administrator elevated cmd prompt
• Go to the installation directory for System Center 2012 Orchestrator on the Management Server

Example: C:\Program Files (x86)\Microsoft System Center 2012\Orchestrator\Management Server>

• Execute the following ensuring you are putting in your Group and “primary user” to manage things.  Both need to be on this command line or you will end up overwriting with what you put here (leaving only group and no user or vice versa)

PermissionsConfig.exe -OrchestratorUsersGroup "DOMAIN\GROUPNAME" -OrchestratorUser "DOMAIN\USERNAME" –remote

Note: The above command is word-wrapping for readability – this command is executed all on one line.  Also, be careful not to “copy / paste” using the example from the TechNet article below (or even this blog for that matter).  Ensure that the “-“ between the commands are in fact dashes and not “non ascii” characters, or the command will fail. 

Additional Note: For additional troubleshooting information on this command (success / failure) navigate to the following directory on the Management Server and review the permissionsconfig.exe*.logs.

%SystemDrive%\programdata\Microsoft System Center 2012\Orchestrator\PermissionsConfig.exe\logs

https://technet.microsoft.com/en-us/library/hh463588.aspx

Below is a abstract of what the command line options mean for your reference

Short, simple, and potentially very useful!  As always –  Happy Automating!

Some Background

To give you a little background on the group I work in at Microsoft currently;  MPSD (Managed Platforms and Service Delivery) supports ~300,000 client systems within Microsoft.  On occasion we are required to pull logs from client systems to troubleshoot various items related to the System Center products we dogfood and support.  Previously, our go to mechanisms for collecting logs have been somewhat numbered and all called “Log Copy”, but utilizing different approaches.

• ConfigMgr Software Distribution – static log collection script targeted by advertisement
• PSEXEC scripts to run remotely against machines and gather these logs
• VBSCRIPTS, PowerShell scripts, batch files

You get the idea…lots of ways, but all somewhat limited in one way or another.

Negatives to the Above Solutions

• Log collection not possible if ConfigMgr client is broken (if using the Software Distribution method)
• File compression wasn’t always possible and likely not used to keep complexity down between different OSs
• Multiple solutions and multiple folks supporting
• Difficult to maintain and keep up to date
• Not scalable – single threaded grabbing one log from one machine at a time
• Manually driven and cumbersome in most cases
• Restricted to few users due to rights required

Introducing the System Center Orchestrator Log Copy Workflow

The Log Copy Workflow, in its simplest form, is a front end interface built on PowerShell and rolled into an EXE that instructs Orchestrator on the back end to gather logs on behalf of the requestor.  The UI accepts requests for scenario based logs (predefined logs) or custom logs (logs you choose with restrictions) and runs these requests against the machines that are provided.  This UI, when the requestor hits the “Submit” button, will generate an XML request file that gets copied to a file share on our System Center Orchestrator Runbook (Action) Server.  Once there, our Runbook automatically picks up this file, processes the data within this file as a request that is submitted into the system.  This request is inputted into a backend database table which Orchestrator leverages to know what “work” it has to do.  This same table is utilized for a retry queue that happens every 2 hours by another Runbook executing on a timed interval.

Front End

The front end shown below is essentially the portal to request logs.  You add machines from an input file or from the clipboard.  You can choose a scenario from the drop down by selecting check boxes as shown and they show up in the list.  A custom log request is also available.  By default, the front end is launched and prepopulates a majority of the look and feel from a XAML file stored on our Runbook (Action) server.  The email address is used to isolate who is requesting the logs (pulled from logged on user ID).  The email field can additionally be semi colon separated to include multiple recipients to be notified each time an email is sent on status of log collection.  A description field is provided and required in order to notate what the log request was for (shows up in the subject line of the email you receive).  Finally, you can set a retry interval (maximum of 99) which will allow you to request a retry to occur by default every 2 hours for a maximum of 99 hours to capture machines that are offline at the time of your initial request.

Notifications

As mentioned above, an email notification is sent to provide status of your log copy request.  This is done by default at the beginning of each request, and the completion of each request (grouped together by request so you receive only one email for all machines you have requested), and if you select the retry copy option, you will receive an email by default every 2 hours until all systems have successfully been gathered or until the retry has expired in number of hours.  Example below.

What’s Next?

Well, this was just an introduction to this solution.  Stay tuned for a more in depth analysis of the Log Copy Workflow within Orchestrator.  For now, I hope you’ve gotten a taste of one potential scenario that can leverage the power of System Center Orchestrator for IT Process Automation.  This solution has definitely provided impact and value for our organization and has shown how automation can shine when applied to repetitive tasks to save time and effort.

Giving “Props”

I love giving credit where credit is due and ensuring the folks I work with get noticed for their hard work.  I have a very talented developer working on my team that works with me to make these workflows happen.  David Wei deserves credit for his hard work and effort on putting together my design, architecture, and overall ad hock requests into our finished products.  Great teams make great end results – thank you David for all your hard work on this project and continued support as we move forward with more automation!

That’s it for now – Happy Automating!

1. Provide an easy way to request data from an end user
2. Make it familiar and intuitive
3. Provide the data in a format that could be easily consumed by Orchestrator via PowerShell.

Solution?

I opted for a quick and dirty Excel XML file.  Looks exactly like an EXCEL XLS file but based on XML data format.  The benefits from this approach were two fold.  I didn’t have to program a fancy front end interface (more on this in a future blog post however ), and PowerShell could consume this data and immediately turn it into Published Data for the Runbook processing within Orchestrator.

STEP 1

So where do you start?  Start with a basic Excel XML file.  The easiest thing to do is put columns together as shown below that frame up what you are looking to gather.  Highlights can be leveraged as shown to call attention to required data.  In addition, you can even put in some data validation (three letters required, etc.) as well as “hover text” to provide your end user the ability to review helpful tips as they are filling out the required data.

STEP 2

Now that you have a basic template file put together, you can either have the user drop it into a pre-determined (monitored by Orchestrator) input directory and allow automation to pick up this file and appropriately process it.  Or what we did was attach it to a service ticket (in our case Team Foundation Server) and allow Orchestrator to collect up that file from the ticket, process the data inside, and act accordingly (updating the ticket along the way of progress).

STEP 3

So far so good right?  Next, once you have an idea of where you are going to place the file for processing, leverage a simple Run.NET object within Orchestrator to process the file according to cell and row analysis.  The code snippit below basically sets variables in PowerShell according to text values located within the called out row and cell. Then you set returned data within your Runbook to the PowerShell variables.

Example: line 9 below evaluates $table.Row[1].Cell[1].Data."#text"

The value shown in row(1) and cell(1) is “SEC”.  In contract row(0) cell(1) is “Values”.  So for each row and cell combination you have with data you need, you will set to an appropriate variable that you can leverage within your Runbook inside Orchestrator.

 1: #Setup the file and initialize Excel COM object

 2: $file1 = "{Published Data}"

 3:  

 4: #Read in XML Data

 5: $template  = [xml](Get-Content "$File1")

 6: $table = $template.Workbook.Worksheet[0].Table

 7:  

 8: #Read values and set variables

 9: $DeployType        = $table.Row[1].Cell[1].Data."#text"

 10: $DecomSvr           = $table.Row[2].Cell[1].Data."#text"

 11: $DecomVar           = $table.Row[3].Cell[1].Data."#text"

 12: $NewSvr               = $table.Row[4].Cell[1].Data."#text"

 13: $NewSite              = $table.Row[5].Cell[1].Data."#text"

 14: $ParentSvr           = $table.Row[6].Cell[1].Data."#text"

 15: $CNTSVR              = $table.Row[7].Cell[1].Data."#text"

 16: $NetworkBin         = $table.Row[8].Cell[1].Data."#text"

 17: $LocalBin              = $table.Row[9].Cell[1].Data."#text"

 18: $dpGroupOSD      = $table.Row[10].Cell[1].Data."#text"

 19: $Install_directory = $table.Row[11].Cell[1].Data."#text"

 20: $PKGID                 = $table.Row[12].Cell[1].Data."#text"

 21: $DP_Drive            = $table.Row[13].Cell[1].Data."#text"

 22: $emailAddress      = $table.Row[14].Cell[1].Data."#text"

STEP 4

Last and final step is to set Published Data within your Runbook to the variable data you have set in your Run.NET object.

That’s it!  I’ve provided an example input file and PowerShell script for you to review and play around with.  If you have any questions – as always please don’t hesitate to ask!  Thanks for stopping by and till next time, Happy Automating!

What is so important and why?

We will all need a way to report on what policies in Opalis are currently in process of development and are not yet checked in.  Even more important, where they were last checked out and who checked them out last!  If they are checked out, they are not saved in the database and will be lost on migration.

Why is this particular topic is so important you ask?  Well, as part of your migration strategy to Orchestrator from Opalis,  you should have in it your plan to export all “key” workflows and sub workflows so they can be imported into your new Orchestrator environment.  You can browse through your Opalis Client and look at workflows one by one and see a padlock on them indicating they are checked out remotely or a pencil if they are checked out from your own machine.  The padlock ones are the tricky ones.

So what can we do to find all checked out Policies?

It just so happens that some of this data is stored directly in the Opalis Database.  Once you get this data, you need to do some manipulation of some data and then extract the data.  So what better way to do this than…you got it create another workflow to automate it.  I’m going to keep this simple and straightforward and you can take it from here and make it as complex as you want.  It is just three simple activities linked together as shown below.

Query Database Activity

1. Start simply by creating a new Policy and drop a Query DB activity on the designer palette.  The query DB object I am using is one of the foundation objects that came with Opalis found under the “Utilities” section on the right hand side of the designer.
   
2. Open the Query Database activity and select the “Connection”
3. Ensure you are selecting the appropriate DB type and set your SERVER and CATALOG (Opalis would be the default catalog).
   
4. Next, select the “Details” tab and type in the following query
   

Get Account from SID Activity

1. Next, I call this the “Get Account from SID” activity but it is doing a bit more than that.  This activity is setting published data for each item you have above and doing some extra PowerShell manipulation.  Drag a “Run.net” object from the “System” section of the foundation objects on the right hand side of the client.
   
2. Link the two objects you have together and place the PowerShell code you see below into the details section (ensuring you are setting it to PowerShell for the type). 
   
   An “eye chart” I know – I provided the workflow at the end of this blog so you can easily recreate and modify.  :)
   
3. One very key step that I want to note above is the gathering of the UserID from the SID (see below).
   
   

      1: if ($lastModifiedBySid -ne "NULL")

      2: {

      3:     $LastModifiedBy = (new-object system.security.principal.securityidentifier($lastModifiedBySid)).translate([system.security.principal.ntaccount])}

      4:     $CheckOutUser = (new-object system.security.principal.securityidentifier($checkedOutUsersid)).translate([system.security.principal.ntaccount]

      5: )



4. Finally, we need to established published data objects for each variable we have defined within our PowerShell script.  This gets us all our published data that can be leveraged in the next step. 
   

Append Line Activity

1. Next, this is the step that probably lacks the coolness that the rest of the process above clearly has .  I grabbed the "Append Line” object from the foundation objects list out of the “Text File Management” section (sometimes a hard one to find).
   
2. Type in the path you want to save your file to (I picked the temp folder on my Action Server) and then populate the published data you care about.
   

That’s it!  Check it in and run it. 

Your results will look something like the following (hopefully you don’t name all your policies “New Policy”)

Final note: The download is provided as an example – use at your own discretion and test first in a lab environment or sandbox to see results.

Thanks for following along – if you want to cut to the chase, just download the provided file and import it into your environment, configure the appropriate pieces, and let it fly.  I hope it helps with the migration.

Till next time…

Happy Automating!

So….What’s in it Anyway?

Provided for your convenience, here is a sneak peak at what content is in this guide.  It is chock full of information that was gathered during our initial deployments internally.  Some of this follows along with Charles Joy’s great video’s available on his blog so go check those out as well!

Table of Contents

1. Infrastructure Requirements for Opalis

2. Installing the Opalis Core Components

2.1. Installing the Management Server

2.2. Creating and Configuring the Database for Opalis

2.3. Importing the License information for Opalis

2.4. Upgrading to 6.3

2.5. Deploying Action Servers from Deployment Manager

2.6. Deploying Clients from Deployment Manager

3. Installing the Opalis Operator Console

3.1. Downloading the Pre-Requisites

3.2. Preparing for the Console Install

3.2.1. Extract the Opalis Operator Console Installer

3.2.2. Install and Configure JDK 6 Update 4

3.2.3. Extract and Install JBOSS Application Server

3.3. Setup and Configure the Operator Console

4. Running the Operator Console

4.1. Running the Operator Console Interactively

4.2. Running the Operator Console as a Service

5. Manual Installation of Opalis Client

That’s it! Short and simple.  If you have any questions, please do not hesitate to reach out and I will do my best to answer or get additional information for you.

And as always,

Happy Automating!

Setting the Stage

To set the stage for the issue, I need to give you a little background on how the problem surfaced to begin with.  This is an interesting one for sure.  There are two things I really like about Opalis and Orchestrator.

1. You can “tell a story” with your activities and runbooks.  Each individual activity (ex: Run.Net object) is like a paragraph in a chapter.  The runbooks I guess would then be chapters.  Following this analogy through to the end, multiple runbooks combined together gives you a book (or novel if you prefer ).    
2. In addition to being very descriptive on how you put the pieces together on the designer palette, you can also put descriptions on your links in between these activities which can provide additional information on what may be happening next, etc.

If you notice in the example below, the links between the objects can have descriptive names such as “Success” or “Failure” or “Go” – essentially anything you want in there.  The default is the word “Link” or whatever you have changed it to.  You cannot have simply a “blank” by inputting  a blank in the GUI and hitting enter.  It will default back to the previous text that was there.  Me being the type of person that wants to ensure my policies are visually appealing I leveraged a “hack” (CNTRL+SHIFT+6) to make the link text disappear and we could have essentially a single “underscore”.  This was something I picked up in an training I was in.  This may have proliferated out as it was a great way to clean up your policies so they were more visually appealing.  See below for an example.

So what’s the issue?

So the issue comes into play when you want to import your exported Opalis 6.3 workflows into Orchestrator Beta 2012.  The XML processing logic that is leveraged by Orchestrator in the new version is not allowing for CHAR(1) through CHAR(31) to be processed.  Long story short, CNTRL+SHIFT+6 is in this list of non printable (non-allowable) characters.

How do you know if you have this issue?

So working with Charles Joy,  we (mostly Charles) created this SQL query that will find all offending CHARs that Orchestrator has an issue with and display them in the results to take action on.

The above query yields something like what is displayed below.  It will show you the offending CHAR value  (in this case it is the CNTRL+SHIFT+6) along with the Policy Name that you can find the problem child.

So how do I update this you ask?

Recommendation 1:

Manually.  Find all occurrences of the non printable CHARs that are unsupported with the query above, chase them down from there to update your Opalis workflows before you go and export them to re-import them into Orchestrator.

Recommendation 2:

Don’t like recommendation 1 above?  Well, there is another couple of queries that Charles Joy and I worked out for updating our MPSD Opalis lab environment.   Two options are presented below because we were trying to take into consideration scenarios where you may have the special character at the beginning of a string or in a case where it is the only thing on a line.

Note: Run this against a backup DB before running in production.  This is a “use at your own risk” type thing.  This isn’t reversable so make sure you are testing, retesting, and validating your tests.

That’s it.  The SQL queries are provided in a download for your review and testing.  And I mean it…..test!

Happy Automating!  Enjoy Orchestrator!

Details

This webcast concentrated specifically on the following details:

• Real world scenarios we are currently leveraging Opalis for
• Service Request Automation (overview)
• ConfigMgr Role Deployment (Demo)
• Patch Tuesday Automation (Demo)
• Reach Analysis for Opalis at Microsoft (overview)
• Architecture considerations
• Single Management Server and Single Action Server
• Single Management Server and Multiple Action Servers
• Lessons Learned
• Moving from simple to scalable / componentized workflows
• Leveraging paper specs to build out initial workflow design before development efforts

This webcast was well received and we believe is a solid hour of great information on how you may look at implementing Opalis and Orchestrator into your own environment.  To view the webcast on demand click on the following link: https://bit.ly/iuZz2h

Q & A from the presentation

Q: How far along in your process did you realize you needed to go into componentization?

A: Almost immediately. Our reach workflow showed that if we hadn’t componentized our workflows, they wouldn’t be very scalable or efficient.
(Note: see next Q/A for more information on this).

Q: What kind of requirements did you find concerning the number of action servers required to run concurrent policies? Meaning did you need to deploy a certain number of action servers to run the necessary number of simultaneous policies?

A: The answer to this question is, it depends.  To put a finer point on this, the Desktop Management side within MPSD is essentially supporting ~280,000 systems worldwide on a single Opalis Management Server and separate Action Server (both running on Hyper-V as guest VMs).  The reason the answer to this question is “it depends” is mostly due to the fact that there are many ways to slice out how you may execute and design runbooks to accomplish automation in your environment.  Do we hit all 280,000 systems from our environment directly from Opalis – no.  We indirectly manage those systems through the use of System Center Configuration Manager (and the System Center suite as a whole for that matter in one way or another).  Key detail on how you design your architecture really has to do with what you are doing and how you are going to do it.  If you have lots of long running tasks that take several hours to complete, you may be leveraging your Action Servers in a way that will prevent you from scaling efficiently (long running tasks taking up memory and running over long periods of time reducing your ability to push more policies into the queue).  Building scalable workflows is an important piece of understanding how to optimize your Action Server load.  Final note on this is – mileage may and will vary depending upon what you are doing.  Analyze how  you are executing and what you are executing in regards to your IT Process Automation efforts.  If you can reduce your runbook tasks in to subrunbooks that execute quickly and return back to a main orchestration workflow, you will spend less time waiting on many tasks to complete in a larger single threaded workflow and allow Opalis to complete quicker and more efficiently allowing you to run more workflows on less infrastructure.

Q: Would you say that action server architecture decisions are more location-based than concurrent policy based? So, in deploying additional action servers in a high volume location, then deploying an action server in a location that might have a high latency link with the main datacenter/Opalis Management Server location?

A: The answer is both :).  It depends on how you want to spread the load of your policies and whether different locations are sensitive to certain infrastructure needs (non trusted domains,etc.).  Action Server build out can be useful to allow you to have more policies running concurrently but more often it is geographical (as long as your runbooks are built in a scalable fashion).

As always, any questions – hit me up! 

And of course – Happy Automating!

https://blogs.technet.com/b/system_center_in_action/archive/2011/06/24/opalis-microsoft.aspx?wa=wsignin1.0

Happy Automating!