MSBuild not building all targets in true parallel manner

Mohammad Mohsin Siddiqui 175 Reputation points
2023-03-08T12:33:54.6733333+00:00

Let me first brief the structure of our project.

We have a single CMakeLists.txt which has multiple library targets (added via add*library) and multiple exe targets (added via add_*executable) and each exe has multiple dependencies of libraries defined in same CMakeLists.txt.

We are statically linking libraries to exe. And our libraries are defined in layerd manner. So to access any lower layer artifact we do include lower ONE master header which includes all headers of the lower layer. For every layer/library, we are using target_precompile_headers like below

target_precompile_headers(UpperLayerLib Private <LowerLayerHeaderPath>/LowerLayerLib.hpp)

For Windows build, we are using Visual Studio Generator. So when CMake Cache generation is completed it in terns creates a sln (ProjectName.sln) and multiple vcxproj files for each target created via add_library, add_executable, etc..

To build our target we are using cmake like below.

  1. Building a single library -> This builds find and I see it uses all cores cmake --build --preset=<BuildPresetName> --parallel --target LibA -- /p:CL_MPcount=<logicalCores> and above command actually runs the below command using MsBuild C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe /m /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /p:CL_MPcount=16 /v:m LibA.vcxproj
  2. Building exe target with multiple dependent target: This builds find and I see it uses all cores cmake --build --preset=<BuildPresetName> --parallel --target ExeTarget-- /p:CL_MPcount=<logicalCores> and above command actually runs the below command using MsBuild C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe /m /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /p:CL_MPcount=16 /v:m ExeTarget.vcxproj What I found in this it started building all dependent targets in parallel. And I could also see multiple instance of MSBuild and cl.exe in task manager. Also noticed that till build complete it uses almost 100 CPU.
  3. Building multiple independent library targets : This one seems that not building in true parallel. cmake --build --preset=<BuildPresetName> --parallel --target LibA;LibB;LibC;LibD -- /p:CL_MPcount=<logicalCores> and the above command actually spawns 4 MSBuild instance and started building with logs like below
   C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe /m /m /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /p:CL_MPcount=16 /v:m TWDevSupport.vcxproj
   MSBuild logs and debug information will be at "C:\Users\mohsin.siddiqui\AppData\Local\Temp\MSBuildTempmohsin.siddiqui\MSBuild_Logs"
     Checking Build System
     Building Custom Rule D:/TW/Workspace/CMakeLists.txt
     cmake_pch.cxx
    <bunch_of_cpp_files>
     TWDevSupport.vcxproj -> D:\TW\Builds\TWLib\Windows-x64\Debug\TWDevSupport.lib
   
   C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe /m /m /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0 /p:CL_MPcount=16 /v:m TWUniversal.vcxproj
   MSBuild logs and debug information will be at "C:\Users\mohsin.siddiqui\AppData\Local\Temp\MSBuildTempmohsin.siddiqui\MSBuild_Logs"
     Building Custom Rule D:/TW/Workspace/CMakeLists.txt
     cmake_pch.cxx
    <bunch_of_cpp_files>
     TWUniversal.vcxproj -> D:\TW\Builds\TWLib\Windows-x64\Debug\TWUniversal.lib
   
   C:/Program Files/Microsoft Visual Studio/2022/Professional/MSBuild/Current/Bin/amd64/MSBuild.exe /m /m /p:Configuration=Debug /p:Platform=x64 /p:VisualStudioVersion=17.0  /p:CL_MPcount=16 /v:m TWCore.vcxproj
   MSBuild logs and debug information will be at "C:\Users\mohsin.siddiqui\AppData\Local\Temp\MSBuildTempmohsin.siddiqui\MSBuild_Logs"
     Building Custom Rule D:/TW/Workspace/CMakeLists.txt
     cmake_pch.cxx
     <bunch_of_cpp_files
     TWCore.vcxproj -> D:\TW\Builds\TWLib\Windows-x64\Debug\TWCore.lib
   ========= Build Successful =========
   Elapsed time: 9586 Milliseconds
   

We can easily see in the above logs that targets are not built in parallel. What arongI am doing is that my independent targets are not built in parallel.

If I build multiple targets selecting via project.sln (via Visual Studio IDE ) it works as expected.

Any help would be greatly appreciated.

Developer technologies | Visual Studio | Other
{count} vote

Your answer

Answers can be marked as Accepted Answers by the question author, which helps users to know the answer solved the author's problem.