Partager via

Recursion, and ItemGroups inside Targets

  • Article

Greetings MSBuilders!

An internal team (Office Live, I think) asked us how to make their build start up faster. Their traversal project at the root of their tree started like this:

<Project xmlns="https://schemas.microsoft.com/developer/msbuild/2003">
  <ItemGroup>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Databases\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\External\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Library\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Tools\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Utils\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\WebControls\**\*.csproj"/>

    <DotNetProjects Include="$(BuildPath)\**\*.csproj" Exclude="@(ExcludedDirectory)"/>
    <NativeProjects Include="$(BuildPath)\**\*.CPPproj" Exclude="@(ExcludedDirectory)" />
    <WebProjects Include="$(BuildPath)\**\*.webproj" Exclude="@(ExcludedDirectory)" />
    <WixProjects Include="$(BuildPath)\**\*.wixproj" Exclude="@(ExcludedDirectory)" />
    <LocProjects Include="$(BuildPath)\**\*.locproj" Exclude="@(ExcludedDirectory)" />
  </ItemGroup>

...

The way they had this set up, was their developers to run a shared batch file at any point in the tree. It would set an environment variable BuildPath to the current directory, and then build this shared traversal project. The traversal project would find every project below the current directory, and build it. This allowed them to avoid putting traversal projects at every point in the tree and remember to add new projects to them. But the wildcard evaluation was slow.

They changed it to this

<Project InitialTargets="FindProjects" xmlns="https://schemas.microsoft.com/developer/msbuild/2003">

  <ItemGroup>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Databases\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\External\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Library\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Tools\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\Utils\**\*.csproj"/>
    <ExcludedDirectory Include="$(BuildPath)\**\Common\WebControls\**\*.csproj"/>
    <AllProjects Include="$(BuildPath)\**\*.*proj" Exclude="@(ExcludedDirectory)" />
  </ItemGroup>

  <Target Name="FindProjects">
      <ItemGroup>      
       <DotNetProjects Include="@(AllProjects)" Condition="'%(AllProjects.Extension)'=='.csproj'" />
       <NativeProjects Include="@(AllProjects)" Condition="'%(AllProjects.Extension)'=='.cppproj'" />
       <WebProjects Include="@(AllProjects)" Condition="'%(AllProjects.Extension)'=='.webproj'" />
       <WixProjects Include="@(AllProjects)" Condition="'%(AllProjects.Extension)'=='.wixproj'" />
       <LocProjects Include="@(AllProjects)" Condition="'%(AllProjects.Extension)'=='.locproj'" />
      </ItemGroup>
   </Target>
  
   ...

With the old approach, this took 12.6s to list all projects in the tree to the console. With the new approach target it takes 3.1s to list all projects in the tree. Of course, most of the time, the build starts lower in the tree, so the time taken is smaller. I wonder if they could have reduced the ExcludedDirectory line to just one line to make it faster still?

    <ExcludedDirectory Include="$(BuildPath)\**\Common\**\*.csproj"/>

This example illustrates a new feature you'll find in Orcas Beta 1 -- ItemGroup and PropertyGroup are now allowed inside Targets. (In MSBuild v2.0 you would have to use CreateItem and CreateProperty instead, which were ugly and inconsistent.) I moved the ItemGroup inside the Target here, so that I could use batching to exclude specific extensions from each list.

I'll be blogging in detail about this new syntax in a future post.

Have fun!
[ Author: Dan, MSBuild developer ]

Update: They couldn't have a single ExcludedDirectory line as I suggested above, because one directory in that tree (Common\Platform) needed to be included. So they now do this:

    <AllProjectFiles Include="$(BuildPathActual)\**\*.*proj" Exclude="$(BuildPathActual)\**\Common\**\*.csproj" />
    <!--We need to re-include the platform directory, so that it gets built-->
    <AllProjectFiles Include="$(BuildPathActual)\**\Common\Platform\**\*.csproj"/>

Which takes it down from 11 to 3 traversals.

Comments

  • Anonymous
    May 15, 2007
    Cannot wait to see ItemGroups in Targets! It should be implemented much earlier!

  • Anonymous
    January 12, 2009
    how can i build a .csproj using msbuild ? when am giving the following code its saying no inputs specified. <Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003"        DefaultTargets="CompileAll"        ToolsVersion="3.5"  >  <Import Project="$(MSBuildToolsPath)Microsoft.CSharp.targets" />  <!-- Import Project="c:.net3.5Microsoft.Csharp.targets" /    -->  <PropertyGroup>    <!-- This AppName thing is the base name of your DLL or EXE -->    <AppName>winApplication</AppName>  </PropertyGroup>  <!-- This build file compiles each .cs file into its own exe -->  <PropertyGroup Condition="'$(Configuration)'==''">    <Configuration>Debug</Configuration>    <!-- Default -->  </PropertyGroup>  <PropertyGroup Condition="'$(Configuration)'=='Debug'">    <Optimize>false</Optimize>    <DebugSymbols>true</DebugSymbols>    <!-- <OutputPath>.bin</OutputPath>  -->    <OutputPath>.</OutputPath>    <OutDir>.</OutDir>    <IntermediateOutputPath>.</IntermediateOutputPath>  </PropertyGroup>  <!-- Specify the inputs by type and file name -->  <ItemGroup>    <Project Include="path*.csproj" />    </ItemGroup>  <!-- specify reference assemblies for all builds in this project -->  <ItemGroup>    <Reference Include="path*.dll" />  </ItemGroup>  <Target Name="FindProjects"> </Target>  <Target Name="CompileAll" DependsOnTargets="ResolveAssemblyReferences" >    <Message Text="Reference = @(Reference)" />    <Message Text="ReferencePath = @(ReferencePath)" />    <!-- Message Text="MS Build Tools path:  $(MSBuildToolsPath)" / -->    <!-- Run the Visual C# compilation on all the .cs files. -->    <CSC      Sources="@(CSFile)"      References="@(ReferencePath)"      OutputAssembly="$(OutputPath)$(AppName).exe"      EmitDebugInformation="$(DebugSymbols)"      TargetType="exe"      Toolpath="$(MSBuildToolsPath)"      Nologo="true"        />  </Target>  <!-- redefine the Clean target, from the Microsoft.csharp.targets file.  (Last definition wins) -->  <Target Name="Clean">    <Delete Files="$(OutputPath)$(AppName).exe"/>    <Delete Files="$(OutputPath)$(AppName).pdb"/>    <Delete Files="%(CSFile.identity)~"/>    <Delete Files="build.xml~"/>  </Target> </Project> can u figure out where the problem lies?

  • Anonymous
    February 09, 2011
    Interesting usage of the ItemGroup/Item concept with Conditions, I have a similar problem in that I need to include all files from a hierarchy of folders is a particular file exists in the hierarchy, I am trying to do this like this: <Files Include="$(SnapshotDir)" Condition=" Exists('$(SnapshotDir)_gac.txt') " /> However the resulting Files is always NULL why is that?