Discussion Items:

• Overview
• TFSBuild DropLocation
• Calling TFSBuild Activity Multiple Times
• Preventing DropLocation Corruption
• Conclusion

Overview

In Team Foundation Server 2010 (TFS2010), the UpgradeTemplate.xaml allows you to call a TFSBuild custom workflow activity that utilizes the same build scripts, properties, etc. used in Team Foundation Server 2008 (TFS2008)  to compile your projects and solutions.  You may find yourself wanting to make different versions of the UpgradeTemplate.xaml to be able to make multiple calls of theTFSBuild workflow activity within the same build, and may wonder how you can leverage the same drop location.  As you will see, this requires a little extra work than you might expect. 

TFSBuild DropLocation

When writing team build scripts in Team Foundation Server 2008 (TFS2008), we relied on the $(DropLocation) property to know where to handle the output of the build. For example, we may copy various 3rd party assemblies there, do some configuration file changes, restructure the output, and any variety of changes to this final build location.  By default, the drop location starts off as the share location passed in to the queue build window: 

The DropLocation property is captured internally and created within the Microsoft.TeamFoundation.Build.targets file (i.e. C:\Program Files (x86)\MSBuild\Microsoft\VisualStudio\TeamBuild\Microsoft.TeamFoundation.Build.targets).  So when the DropLocation is first constructed it starts with \\server\share and is pulled from the TFS database using the GetBuildProperties custom task within the InitializeBuildProperties target:

You can override the build number that gets added to the end of the original drop location using the skeleton target BuildNumberOverrideTarget that gets called after initializing these properties, but before concatenating the original $(DropLocation) property with the $(BuildNumber) property forming the new updated $(DropLocation) property: 

In the InitializeEndToEndIteration MSBuild target, the original $(DropLocation) is added to the $(BuildNumber) and is then updated in the TFS database:

 This is how your first build of the day would end up with the $(DropLocation) property being: \\server\share\MyBuildDefinition_20110407.1.  The $(BuildNumber) piece is the part in yellow.

Updating Drop Location Using Internal Targets

In MSBuild scripts, you can update the final drop location by either overriding the BuildNumberOverrideTarget and creating different properties like this:
(in TFSBuild.proj or other imported .proj file)

Since my team project is brandoh, this would create a droplocation such as:  \\server\share\brandoh_MyBuildDefinition_20110407.1  . 

Updating Drop Location Using Custom C# Tasks 

The next challenge might come if you want to do some sort of special modification to the droplocation utilizing values derived from some custom task that you wrote.  In this case, this may not work because the assemblies containing your custom tasks will not exist in the workspace yet, because these internal targets take place before the get (before downloading files from source control).  In this case you need to update the drop location registered in the database, after the get.

You may have noticed the AfterGet target skeleton target, which is an empty target that you can override which gets called in the Microsoft.TeamFoundation.Build.targets file.  Since, I have shown some example custom targets to run (RunTarget1, RunTarget2, and RunTarget3), I will use RunTarget1 to demo how to perform an additional custom update:

Basically, I got today's date and created a new date-based folder to store builds in and then updated the database with the new drop location. 

Important: You may wonder why the $(DropLocation) property is the original droplocation in this context and not the one modified earlier in the InitializeEndToEndIteration target.  The answer to this question is somewhat complex, but basically, custom properties only have a scope that pertains to the group of targets within a DependsOnTargets target.  Using the AfterGet example above, the RunTarget2 and RunTarget3 targets will have access to custom properties created in RunTarget1, but no other targets will have access to any other property of other MSBuild.exe target calls, unless they are passed in internally via the command-line (i.e. msbuild.exe targets.proj /p:DropLocation=[ModifiedDropLocation]) or through an MSBuild custom task call.  When Team Build re-enters it's own internal targets, it loses scope of modified properties not pulled again from the database.

Solution: So how do you gain access to modified properties when they lose scope through-out different portions of the build?  This is a great question!  What I recommend as a possible solution to this is to have a simple InitializeVariables target that you call at the beginning of all DependsOnTargets target lists like this:

As you can see, when the build stops compiling solutions/projects, Team Build calls the AfterDropBuild target which will start by running the InitializeVariables target to setup any custom properties derived from custom tasks.  Now, these properties will exist in their correct form throughout all other targets in that DependsOnTargets attribute (RunTarget4, RunTarget5, and RunTarget6).

CustomPropertiesForBuild: You might be wondering what this property is.  This property is an internal property that when you group your property = value semi-colon delimited pairs within it (i.e.  a=1;b=2;c=3, etc.), that property gets passed throughout many of the internal targets via msbuild calls (i.e. BeforeCompileSolution, AfterCompileSolution, etc.) and makes them even more available.

Calling TFSBuild Activity Multiple Times

Everything discussed so far also applies to TFS2008 as well as the UpgradeTemplate in TFS2010.  So what happens if you want to create a new version of the UpgradeTemplate.xaml in TFS2010, where you call the TFSBuild workflow activity more than once?  (i.e. when trying to run different builds in parallel or trying to run TFSBuild multiple times in the same build but in different ways each time)

What happens is that the DropLocation will get corrupted, because it will retrieve the current value in the database and then use it to create a new value appending the build number again.  You will start seeing repeating build numbers like this:   \\server\share\brandoh_MyBuildDefinition_20110407.1\brandoh_MyBuildDefinition_20110407.1\brandoh_MyBuildDefinition_20110407.1.

Preventing DropLocation Corruption

There is hope!  One way to prevent DropLocation corruption is by first capturing the "original" droplocation before adding the build number and then passing it into the TFSBuild workflow activity teaching it to correctly determine it's droplocation based on what was passed in, NOT based on what is internally pulled from the database.

Before we can pass in the original drop location into the TFSBuild activity we first need to get the original droplocation in the workflow.  You can create a variable to capture the original drop location:

Then use the assign activity assign it the value of BuildDetail.DropLocationRoot at the beginning of the build:

Then you will need to pass this value into the TFSBuild workflow activities you want to consider this original droplocation:

Then you can change your msbuild code for RunTarget1 to set the droplocation based on whether or not this value is passed in.  As you can see, if the droplocation property is empty, it means the value wasn't passed in, so we can let the build update as normal (see RunTarget1 above to compare).  If it's not empty, we can use the OriginalDropLocation value to correctly set the droplocation rather than use the value that is pulled internally from the database in the Microsoft.TeamFoundation.Build.targets file: 

Conclusion

By leveraging the principles discussed in this blog you can effectively manage the drop location in a variety of ways, including reflecting the desired drop location in multiple TFSBuild workflow activities.  The ability to utilize the same drop location in different workflow activities lets you utilize modified versions of the UpgradeTemplate.xaml to employ parallel build activities or to engage entire build processes sequentially but in different ways. 

In the future, if I get a chance, I will try and show how you can do some really cool parallel build processes. ;-)

Discussion Items:

• Overview
• Default Template vs. Upgrade Template
• Template Parameters
  • Multiple Build Definitions Using One Template
  • Adding Your Own Parameters
• Workflow Architecting
  • Executing Commands
  • Display Logging
  • Variables
  • Conditioning
  • Looping
  • Sequence
  • Flowchart
  • Parallel
• Imports
• Conclusion

For a summary of general activities that are available and a brief description of each, you can refer to sites such as:

https://blogs.msdn.com/b/endpoint/archive/2009/05/29/a-tour-on-the-wf4-activity-palette.aspx

In general, getting used to using the workflow approach can be a little unusual and confusing at first, but after playing around with it for a little while, you will find that some previously difficult tasks are made simpler with the workflow approach. Unfortunately, each time you open a new XAML file it reloads the toolbox activities which takes a while before it will show the complete workflow. You can view other available activities by right-clicking on the toolbox and clicking "Choose Items...". Clicking the .NET Framework Components tab will take a while to load the available components.

When filling out expressions in the XAML editor, you will note that you need to use Visual Basic to form expressions. For example, for concatenation, you can use the ampersand character: '&'. For multiple lines, use the underscore character '_'. You can also copy existing items within the XAML editor to other areas within the workflow by clicking where you want to copy the item to and then just paste. It preserves the data from the copied workflow item.

When first using workflow in Visual Studio 2010 team build, you may be wondering whether or not you should use the default template or the upgrade template.

Upgrade Template

If you are migrating a large amount of msbuild code from a TFS 2008 server, then it would probably be a lot easier to just use the upgrade template. The upgrade template will wrap a very simple workflow around a simple msbuild workflow activity which only does one thing: run TFSBuild.proj. When it runs TFSBuild.proj it will pass in all the typical parameters that msbuild will need to use the previous approach for building TFSBuild.proj. The advantage to this approach is that it's fairly simple and straightforward and you can still add as many other activities as you see fit. One disadvantage to this is that you lose some of the flexibility of using workflow activities in between specific team build events previously performed in TFS 2008. For example, if you wanted to perform some special logic before or after compiling individual solutions, you wouldn't be able to use workflow activities to assist with this. All your logic has to be before or after TFSBuild.proj is done. This includes workspace manipulation, build number changes, inserting in-between buildsteps, etc. All solution or project information will still be contained within the SolutionToBuild.proj file(s) as before.

Default Template

If you are starting from scratch, it may be worthwhile to consider development using the DefaultTemplate. You will notice that the template is much larger and more complex than the upgrade template. This is mainly because the logic that would normally be contained in:

C:\Program Files (x86) \MSBuild \Microsoft\VisualStudio \TeamBuild \Microsoft.TeamFoundation.Build.targets

is now contained directly within the DefaultTemplate.xaml. Although this makes the workflow look complicated, it gives easier access to internal functionality that was previously more complicated to change. For example, whenever, you wanted to implement a team build process differently than the way it worked before, you would have to "override" those targets contained within the targets files above by listing the same target with the "overridden" logic inside your own TFSBuild.proj (or imported files).

Furthermore, when using the default template you can change the set of projects or solutions that are built at the build definition level instead of having to check-in solution or project changes into the SolutionToBuild.proj file. This can be nice if you want to build different sets of projects within different build definitions.

One really cool feature in TFS 2010 Team Build is the ability to have multiple build definitions pointing to the same template. You don't have to use the DefaultTemplate or the UpgradeTemplate if you don't want to. In fact, it's recommended that you make your own when you feel the need to customize one of the pre-existing templates. Once you have a template that is flexible enough to work differently based on the parameters that are passed in, then you just change the process tab for the different build definition to point to the same template, but passing different parameters.

When you open a new build definition in team explorer and click on the process tab, you will notice a bunch of different parameters that you can tweak for a build definition. These fields are defined within the template XAML.

When you open up a XAML template file, you will notice three tabs at the bottom of the visual workflow: Variables, Arguments, and Imports. In the Arguments tab, you can specify the input parameters passed into the template for processing from the build definition. As visible on the screen for creating the arguments, you can also specify the types and the default values for this.

You can refer to these parameters throughout your workflow template XAML just by starting to type the name of the parameter within a field that you're using later on in your workflow. Intellisense will pick up parameters.

One of the most basic abilities that is nice to perform is the ability to execute some shell based command. For instance, in msbuild scripts you could use the Exec custom task to execute some command (i.e. run some custom executable). One annoyance with the Exec command is the inability to capture and process the command output directly. Basically, you had to pipe the output to some text file and then read the text file and retrieve the information desired.

Happily, WF4 makes this so much nicer by introducing the InvokeProcess activity. The activity lets you execute any provided executable through the "FileName" field and then pass the executable any arguments you want it to handle (such as a shell command).

The beauty of using InvokeProcess is that you can immediately capture and/or perform some sort of activity-based logic on the output whether it's from the standard output or error output streams.

When you want to display something in Build Explorer when building, you can still display items using the BuildStep custom task when using the default template and calling the build steps inside of msbuild scripts.

When trying to display information in Build Explorer outside of the context of msbuild, you can use workflow activities to display information about the build that is running. There are three simple activities that can help with this:

• WriteBuildMessage
• WriteBuildWarning
• WriteBuildError

WriteBuildMessage will show plain messages in BuildExplorer. It's displayed as pure text unlike the normal BuildSteps you might be accustomed to, which show the green circle. The problem with WriteBuildMessage is that it won't show your messages if you have minimal verbosity set. You have to set the verbosity to normal or higher to see these messages. You will also have to set the importance property for this activity to "high".

WriteBuildWarning will show the yellow triangle/exclamation mark and also won't show with minimal debugging. Once you change this to normal debugging, you will be able to see all the warnings (both custom generated and the warnings generated by the build).

WriteBuildError will show the red circle showing that an error was revealed. These custom generated errors don't stop the build, but they will cause the final status will not show success.

The main problem is if you have a huge amount of warnings and want to eliminate warnings, but also want to see the build messages. You either get your regular build messages and warnings, or you get neither. Fortunately, when you use most activities, they will still show messages with the title or description of the activity you are displaying.

In TFS 2008 Team Build, the scope of properties were very tricky. Most of the time values had to continually be passed around or group in DependsOn blocks to maintain values across targets, but in TFS 2010 Team Build it's a lot easier.

Once you have a template XAML open you can click the Variables tab and create any property with any .NET type by typing the name of the new variable and clicking the drop-menu for the type and selecting "Browse For Type...". You can then search every .NET namespace for the type you're looking for and then click it.

Once you have your variable setup, you can determine the scope. For example, variables created with the "Sequence" scope are accessible in the entire sequence (which is the main activity of the workflow). If you have different activities running on different build machines you can choose the "Run On Agent" scope which is specific to a build machine/agent.

You can give the variable a default value and also assign variables at any step along the way by using the "Assign" activity under the Primitives section in the toolbox.

The full span of .NET conditioning is available in WF4. You have access to switches, if/then and the other typical .NET based branching concepts.

There are also the expected try, catch, exception activities as well. For example, if you throw an exception, it will display the exception and halt the workflow.

There are a couple activities that may seem a little unusual that also help in performing different handling techniques such as handling cancellations or performing recovery if something breaks later on in the workflow.

For example, the CancellationScope activity is a container activity where if you can embed an activity inside the CancellationScope activity. You also provide a CancellationHandler activity. This means that if the workflow is cancelled while running the imbedded activity, then it will call the activity given inside the CancellationHandler portion. This allows for custom recovery of a cancellation.

Another somewhat more advanced, but confusing activity is the CompensableActivity, which will run a given "Body" activity. Based on various circumstances, different handlers will be called. For example, if the activity is cancelled, it will call the activity provided as the CancellationHandler. If the workflow breaks anywhere after the entire CompensableActivity is finished executing successfully, then it will call the activity provided as the CompensationHandler. Otherwise, if the entire workflow completes successfully, then it will call the ConfirmationHandler. A common example for this, is if the "Body" is scheduling a flight. If the flight is cancelled while scheduling the flight, then you just cancel scheduling the flight. If after scheduling the flight successfully, you schedule a hotel and do some other random scheduling. If the flight is successfully scheduled, but let's say the hotel scheduling failed, then you would want to also provide some way to cancel the flight as well since you won't be able to go on the trip anyways.

You can use most of the normal .NET looping structures such as ForEach, While, and DoWhile although there isn't an activity provided for a typical "for" loop.

While and DoWhile are fairly simple by simply providing the body activity and the condition for continuing to execute the body activity over and over, the only difference, of course, is that DoWhile will execute once regardless of the condition.

The ForEach<Type> is generic so you have to configure the properties for the activity to indicate the type of the individual item you will be pulling out of the collection. The first field is to represent the name of the individual item. Inside the expression you just give the name of the collection variable. When you setup the body activity, the item name is within the scope of the body of the foreach activity and can be accessed accordingly.

The sequence activity is known as a composite activity, because it allows other activities to be placed within it. If you ever need to cluster a set of activities inside a "single" activity, you simple drop a sequence inside the "single" activity placeholder and then drop all necessary activities inside the sequence.

Please note that unlike flowcharts, you cannot go back earlier in a list of activities if you have already traversed the list.

Flowchart

Flowcharts are workflow's way of allowing designers to go back and repeat previous steps already executed and navigate more freely directionally.

Here are the results:

It is important to note that you may have to include some additional assembly references so the Flowchart activity can operate correctly (necessary in RTM). If you're seeing the following error:

TF215097: An error occurred while initializing a build for build definition [Build Definition] Cannot create unknown type '{https://schemas.microsoft.com/winfx/2006/xaml/presentation}PointCollection'.

Try adding the following import lines at the top of your .xaml file:

xmlns:swm="clr-namespace:System.Windows.Media;assembly=WindowsBase, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"

xmlns:swm1="clr-namespace:System.Windows.Media;assembly=PresentationCore, Version=4.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"

Also, if you have graphical display issues where the flowchart halts display, try letting Windows (i.e. 7) adjust for best appearance or "Let Windows Choose What's Best For My Computer" in the Advanced System Settings "Visual Effects" section.

The parallel activity is unique because the activities placed within it can be placed next to another, which allows the flexibility to run multiple tasks at the same time.

One interesting application of this is running tasks on multiple build agents. For example, you can put multiple AgentScope activities next to each other which allows the ability to spawn off different activities on various build machines to speed up build performance.

Imports

In TFS 2008 you couldn't easily use msbuild imports because the TFSBuild.proj file would try and import the files before performing the get to pull down files from source control. This sort of scenario isn't a problem in TFS 2010, because you can use activities such as the DownloadFiles activity to run prior to running the MSBuild activity so that these files are available locally before running older logic on the TFSBuild.proj. Of course, this applies only if you're using the UpgradeTemplate.xaml. If you're using the DefaultTemplate, you have more flexibility.

Windows Workflow Foundation (WF4) is a nice integration of all previous build capabilities available in MSBuild 3.5 TFS 2008 Team Build, while adding many useful flow control features that were more difficult if not nearly impossible in TFS 2008.