Response to the feature poll
There were over 80 responses to my recent post asking for feedback on where MSBuild should be heading (if the graph doesn't appear, it's here).
Thank you all for your thoughtful allocations! Let's go through each one in decreasing order of "investment".
1) Debugger. Debugging by gazing at diagnostic logs and putting in <Message> tasks is less than ideal, and I've probably done this more than anyone, but I would never have expected that a debugger would be the #1 request. A while back, one of us hacked up a proof of concept for debugging MSBuild in VS by using this technique, and it worked rather well. Apparently this is a perfectly supported route, and presumably much easier than writing a new debugging engine for VS. If and when we support this publicly, it would most likely be by this route. I hope we can.
2) Converting other project types. Deployment (.vdproj) was the highest mentioned by far, with some other mentions of Biztalk. Some good news here. WiX support integrated into Visual Studio has just appeared in the CTP of Visual Studio Rosario. It's based on the existing Votive plugin for Visual Studio, “productized” into the Visual Studio box. Rob Mensching has some detailed information. Of course, .vdproj isn't the same as WiX. I've passed on the feedback from this blog to the owners of this effort (who also own .vdproj) so they can see how many people would love to see .vdproj be MSBuild based. If you grab the CTP you can let them know exactly what you want via opening Connect issues -- Jeff Beehler's blog has information about how to do this. Open issues there so they get feedback, and vote them up the list :-)
3) Converting the .sln file. No surprise this is a big deal. Visual Studio itself is built by MSBuild, and for that purpose we didn’t use .sln files – we use files named "dirs.proj" to represent nodes in the build tree. Each dirs.proj does little more than list the projects and dirs.proj’s below it, and import simple build rules that use the <MSBuild> task to continue the tree traversal. Building Visual Studio itself from a huge solution file would be impossible. So we’re well aware how much pain they cause outside Microsoft. The hard part here from the MSBuild point of view is getting VS to read and write these dirs.proj files instead of .sln files.
4) Visualization. This was the other big surprise for me as I figured that you would say "complete more of the basics first". Also, of the items on this list, this would be one of the easiest for folks outside of Microsoft to accomplish. A couple months ago a couple of us spent a week and prototyped a WPF visualizer. Chris wrote a SQL logger that listened to just the regular, standard logger events and wrote to a database using the free SQL Server Compact Edition. Cliff wrote a visualizer based on the free GLEE graphing package which queried the database. It had two views -- bubble and line of target dependencies, and bubble and line for project dependencies. We built parts of VS using multiprocessor support (“/m” switch) with this logger attached, then pointed the visualizer at the database and we could see, for example, projects causing bottlenecks in the build. Because the visualizer was based on a database, there was some UI to do simple queries, like color all projects with names matching pattern X. Of course, this could go a lot further. What sort of visualizations would you want to see?
5) Native code/VC support. More good news here! Both command line and IDE support is now planned for the next release of Visual Studio after Visual Studio 2008 ("VS10"). Some of our friends downstairs in VC have been working on it for a little while now. As more details emerge we'll pass them on. (Putting my “platform” rather than “Visual Studio” hat on, you may have noticed that earlier this year, a competing IDE product began to offer native code build based on the MSBuild platform.)
6) Distributed build. Inside Microsoft, both the Visual Studio build team (the team that builds VS itself) and the Windows build team have been asking for this, amongst others. Six months ago one of us did a proof of concept of MSBuild based distributed build running on some large subtrees of the VS code and got near linear scalability in some cases - better than linear up to 4 machines in one particularly IO-bound subtree. We have a good idea how we would do this, but it isn't yet scheduled to happen. This seems to be a feature aimed more at the "build lab" than the developer. On the developer's machine, we typically aren't making full use of the cores she already has, and the kinds of builds she does tend to be smaller and perhaps less parallelizeable at the project level and the transmission and provisioning overhead would be significant.
7) Inline tasks. This is one of my personal favorites. Rather than say "go write and compile and deploy a task" I'd like to say "write some Powershell/batch/jscript and paste it in here and use it like a task elsewhere". Again, we've done a proof of concept. One particular kind of inline task we use internally we call "DataDrivenToolTask". You probably know that tasks that wrap a command line tool generally derive from ToolTask. These all have to do the same tiresome job: map friendly properties to command line switches. When we created tasks to wrap the native compiler and linker for building Visual Studio internally, this code became unwieldy. Yet it each mapping is a simple variation of the other. So we threw out the old tasks and created XML files that stated the tool name, defined mappings between property names and switches, described the type of the switch parameter, indicated how the switch is negated, what it could legally be combined with, and so on; then we wrote a generator tool that created the task from these. (Actually it creates a partial class, deriving from ToolTask, which we can combine with a little special code for that tool if we need to.) This turned out to work very well. What I would like to see is the ability to put this XML directly under the UsingTask element, so the task is created dynamically. That's probably not something anyone would do for something as huge as CL, but for tools with just a handful of parameters - think regsvr32 - one could make a nice typesafe readable task and pass it around to your friends by simply pasting it into an email. It's my hope that this would be one of our first types of inline tasks.
8) Multiproc performance. We're embarking on a significant rewrite of the "back end" of the MSBuild engine in VS10 and expect that we'll be able to improve the rather "basic" multiproc scheduling that MSBuild does in v3.5/Orcas: both to improve performance on existing machines and work better on the many core machines that will become increasingly common over the next few years. Our expectation is that this refactoring will make extending to distributed build more straightforward in a future release.
Skipping forward a few (the missing #10 probably should have been "add more built-in tasks") I'd like to mention ..
11) Extensible dependency checking. This was a bit confusingly described. There are really two parts to this: support transitive dependencies, and provide an extensibility point for arbitrary dependency checking plugins (such as an assembly interface comparer). The transitive dependency support is really essential for reasonable support of native code. C/C++, IDL, and .RC files amongst others all pull in files indirectly via #include and it's not feasible or reasonable to expect a developer to describe in their build process the full transitive closure of files that are read by their build in all circumstances: yet to do a trustworthy incremental build MSBuild must check the timestamps of all these files. This problem actually arises in many situations, with arbitrary file types. We have been solving this problem internally for the build of Visual Studio itself using a technology we created called Tracker, and it's proven itself to be solid and reliable. The other part is the extensibility point, which is perhaps what most people were actually voting on, and seems less urgent.
12) Assembly versioning support was suggested. Many of us have used Neil’s AssemblyInfoTask. Again, we hear you and there's good news. TeamBuild is planning to have a similar, but official and supported solution to this shipping in Rosario.
14) Object model improvements weren't on my list but probably should be: they were suggested by several people. Today the MSBuild object model is somewhat clunky and certainly incomplete. It was created to do just exactly what Visual Studio needed for VB and C# projects and no more. It mixes up the evaluated and unevaluated states amongst other problems. We expect to considerably improve this OM - not least because VC in VS10 will need it improved – to make it truly generic and powerful. (I already have a small group of people interested in giving detailed feedback on the design, but if you'd like to join them, please send me an email at msbuild@microsoft.com).
My caveat again -- nothing is set in stone until it appears in the final box, (or else is officially announced!) - resources and priorities change, risks to the product are managed, customer feedback sometimes shifts our direction directly or indirectly. In particular, work we need to do to support larger Visual Studio features has to happen. I hope though that you'll get value from my blogging candidly about our ideas and likely plans, while you realize we can’t make firm commitments to particular features at this stage. Enough caveats :-)
Any thoughts?
Dan
"This posting provided AS-IS, with no warranties"
Comments
Anonymous
November 30, 2007
Dan, Has there been any discussion of supporting remote dependencies via HTTP a la Maven? Supporting the ability to pull assemblies from a remote location such as corporate repository or open source repository would be a great feature.Anonymous
November 30, 2007
@Eric, No, I've never heard that request. This would be pulling assemblies that you require but you yourself haven't built, because you haven't changed the code or haven't enlisted in the code? In the MSBuild world, this would be accomplished by writing a task. Our existing ResolveAssemblyReference task doesn't currently have the ability to look in remote locations except via UNC. DanAnonymous
November 30, 2007
<shameless plug> Regarding visualization, there's http://www.codeplex.com/dependencyvisualizer that you may find useful. </shameless plug>Anonymous
November 30, 2007
Dan, Yes, exactly. For example, I have an open source project that I want to use in my application. Today, I download the assembly and place it in a folder, then add a hardcoded reference in Visual Studio to the location. When I use Java and Maven, I add a depenency to a project and a version in my configuration and it handles the rest (going to the repository, downloading the library, downloading any transitive dependencies). This approach also allows you to establish a corporate repository for shared libraries. I looked at the ResolveAssemblyReference task and the concept is similar. The downside with UNC is that repositories cannot be accessed outside the firewall. However, Visual Studio will not know about these assembly references until I build the project though, correct? That would be an issue for anyone who uses ReSharper, CodeRush, or anything else that does incremental compilation of your project from inside VS.Anonymous
November 30, 2007
@Eric, You should be able to write a task that resolves selected items in the @(Reference) list, caching them locally and adding HintPath metadata to the items as appropriate. If this runs before ResolveAssemblyReference then it will be able to find the reference at the hintpath. As for design time support, if you make the ResolveAssemblyReference target depend on the target you created containing your task, then I believe that Visual Studio should be able to resolve references at design time, since it calls the ResolveAssemblyReference target directly at that time. I haven't had a chance to try this though. DanAnonymous
December 01, 2007
Folks, Hi! A recent feature poll on the MSBuild blog seems to indicate that a debugger for MSBuild projectsAnonymous
December 01, 2007
Here is a debugger for MSBuild Projects http://blogs.msdn.com/parthopdas/archive/2007/12/01/visual-debugger-for-msbuild-projects.aspxAnonymous
December 02, 2007
It looks like I'm not the only one who wanted a debugger for MSBuild projects. When the MSBuild teamAnonymous
December 03, 2007
Dan Mosely recently posted a feature poll on the MSBuild blog . Here are the results . DebuggabilityAnonymous
December 04, 2007
I posted few weeks ago about the feature poll of MSBuild team on the MSBuild blog . Here are the resultsAnonymous
December 04, 2007
The Visual Studio Team System User Education Blog on New End to End Articles for Visual Studio Team System...Anonymous
December 07, 2007
The comment has been removedAnonymous
December 07, 2007
The comment has been removedAnonymous
December 07, 2007
The comment has been removedAnonymous
December 10, 2007
The comment has been removedAnonymous
December 17, 2007
@Eric This is very useful -- I'm passing it along to the team build people who are planning to add assembly version support in Rosario. Do you use VS team build? Or would you envision it being available outside team build? DanAnonymous
December 18, 2007
With the release of MSBuild Sidekick v2 , several questions have been asked repeatedly. But what is theAnonymous
December 20, 2007
Distributed builds would be a big deal for us as a game company (yes I voted form them). We have been doing our project builds with MSBuild for quite a while now. Our build process basically looks like this: (1) pull latest version from version control (2) recompile executables (3) batch-export graphics objects from Maya format to our engine format using our own batch-exporter (4) batch-convert textures from .psd to .dds (5) export levels and other game data into a SQLite database (6) pack assets into archives (7) build an installer (8) ftp-upload the resulting files A typical game consists of about 20,000 to 30,000 source asset files and compiles into about 2..7 GB of data. A full build takes up to 11 hours. An MSBuild tool with built-in support for distributed builds could help us to split the critical steps (3) and (4) across several machines without too much hassle. MSBuild is great by the way ;)Anonymous
December 24, 2007
Thanks Dan (and the rest of the team) - I'm glad you found the feedback useful! Presently we're a small shop of about 6 developers writing larger scale applications (distributed client/server, at least 4 tiers) targeted for 10,000+ users. In that sense we're a little unusual - but Team Build is out of our league right now. (Seems to be targed for organizations > 100 devs, etc). The '5 user' edition of team studio is too restrictive for our environment as well. I would envision this working outside of team build - right now we have a build server running custom msbuild scripts which handle source control, file copies, etc and hand off the actual build of the application's individual tiers to (unmodified) Visual Studio solutions and project files. I find the compiler-level solution so appealing because any build framework could potentially hook into it, whether it's MSBuild, Team Studio Build Server, NAnt, etc. Support in Team Studio would be nice, but it still doesn't address the limitation in the .NET compilers in regards to assembly versioning. Right now I visualize the minimum required changes for this to work would be the modifications to the compilers for the version override and a slight adjustment to the MSBuild target files - anything else is a major bonus (ie/ Visual Studio or Team Build support). Hope that answers your question!Anonymous
January 17, 2008
MSBuild has made my life so much easier that it makes me shudder thinking back to the days of NMAKE.Anonymous
March 25, 2008
Are you looking for a visual debugger for MsBuild? It seems like lots of folks are. Looking at the resultsAnonymous
November 20, 2008
Hello everyone. I’m Marian Luparu and I am a Program Manager in the Visual C++ IDE team. Last week IAnonymous
December 08, 2008
What about fixing something that everybody at some point encounters. The use of absolute paths and the related maximum characters error. This is a constant annoyance for us, since we try to honor .NET naming guidelines and try to give products real names instead of acronyms or similar. And windows does support paths longer than 256 so why this limit? The error: Typically with $(FullName) something: The specified path, file name, or both are too long. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters.Anonymous
December 08, 2008
@NTR I wish we could. We are dependent on the support from .NET framework/CLR API's - they are doing some work on it (http://blogs.gotdotnet.com/bclteam/archive/2008/07/07/long-paths-in-net-part-3-of-3-redux-kim-hamilton.aspx) but it is too limited for MSBuild because many Win32 API's don't support long paths. If this sounds like passing the buck, because we're all Microsoft, I wholeheartedly agree. But another way of looking at it is that it would need changes in an enormous number of places right down the stack. Think of all the statically sized buffers in C++ apps for example. So it would take overwhelming agreement from customers that it is worth doing instead of other work. Hope that makes sense, DanAnonymous
December 17, 2008
Exciting perspectives! And there’s an interesting fact - you can get the Editing UI feature as a part of Automated Build Studio installation package (by AutomatedQA). Also there are some other features from the list above. The macros can be used for extending MSBuild projects. Check it: http://www.automatedqa.com/techpapers/abs_vs_msbuild.aspAnonymous
December 18, 2008
For those of you using vc builds (not msbuild), you can use VersionUtil to update the build number as a post build event in your .vcproj files. http://www.insidercoding.com/page/VersionUtil.aspx Feature requests are welcome.Anonymous
February 18, 2009
Hello. I have a bit of a problem and was wondering if anyone could offer a possible resolution and/or lead towards finding a solution to my problem. I'm working with project templates via VS2008 Professional Team. I'm able to easily add the neccessary references to my templated project, but where I run into problems is trying to rename the default class.cs file associated with making a new project. Could anyone tell me how to take my formatted (templated Class1.cs) file and rename it to something else? ie. Class.cs -> MyClassName.cs within the template. I used this template file: <VSTemplate Version="2.0.0" xmlns="http://schemas.microsoft.com/developer/vstemplate/2005" Type="Project"> <TemplateData> <Name>DMIBusiness Objects</Name> <Description>Extended DMI Business Object Assembly</Description> <ProjectType>CSharp</ProjectType> <ProjectSubType> </ProjectSubType> <SortOrder>1000</SortOrder> <CreateNewFolder>true</CreateNewFolder> <DefaultName>DMIBusiness Objects</DefaultName> <ProvideDefaultName>true</ProvideDefaultName> <LocationField>Enabled</LocationField> <EnableLocationBrowseButton>true</EnableLocationBrowseButton> <Icon>__TemplateIcon.ico</Icon> </TemplateData> <TemplateContent> <Project TargetFileName="DMIBusiness Objects.csproj" File="DMIBusiness Objects.csproj" ReplaceParameters="true"> <Folder Name="Properties" TargetFolderName="Properties"> <ProjectItem ReplaceParameters="true" TargetFileName="AssemblyInfo.cs">AssemblyInfo.cs</ProjectItem> </Folder> </Project> </TemplateContent> </VSTemplate> I even tried changing it within the Project.TargetFileName to: TargetFileName="$safeprojectname$.cs"??? HelpAnonymous
March 24, 2009
We are happy to announce the release of version 2.3 of MSBuild Sidekick! Many of you would remember "Anonymous
March 30, 2009
The folks at Attrice have released a new version of their MSBuild Sidekick , and it now includes a visualAnonymous
July 07, 2010
Back when we started 4.0 development, I polled readers of the MSBuild blog to find out what featuresAnonymous
July 07, 2010
Back when we started 4.0 development, I polled readers of the MSBuild blog to find out what featuresAnonymous
July 07, 2010
Back when we started 4.0 development, I polled readers of the MSBuild blog to find out what features