third-party merge tools and exit codes

One of the extensibility points we have is the ability to specify third-party tools for diff (file comparison) and merge (three-way merge specifically) operations.  For merge, when we call the third-party tools and then they return, we have to figure out whether we should accept whether we should trust the resulting file they produced (if it produced a file).

Initially we coded it so that if 1) the file was produced 2) the produced file was not empty (admittedly there are some scenarios where an empty output file is valid from a three-way merge, but that's a very tiny corner case for us) and 3) the exit code was 0, then we accepted the output file.

However, we found at least one tool that would exit with a non-zero exit code even when the user had successfully resolved all the conflicts and saved the merged output, so we modified the code so that criteria #3 became a prompt to the user as to whether they want to accept the merged output file from the tool or not (the default tool we ship with, BTW, isn't treated as special, it's just the default merge tool, but is still treated just like any other third-party merge tool, and is called through the same code and same interface).

Unfortunately, this has now led to a "double prompt" scenario, in that if you resolve all your changes and exit the tool, you're prompted once as you exit the tool (ours included) as to whether you want to save the output merged (this is typical as you exit most apps, of course), but then you get prompted again as we see the tool exited and prompt you for whether you want to keep the output file that the tool produced.

So, now we're considering just prompting when the tool exits with a non-zero exit code.  Since the typical case is that if the tool ran fine, the exit code was 0, this helps take care of the typical case by reducing a prompt, but also allows tools that return non-zero exit codes even when they run fine to have the user keep the output file.

What do you think?  What's approach would you like to see used?  Keep in mind we have to code something that will work for not only the current set of merge tools that exist, but ones that may be written after our release.

Comments

• Anonymous
  June 29, 2005
  Can you trust the meaning of an exit code from an app of which you have no specific knowledge? I wouldn’t...
• Anonymous
  June 29, 2005
  To say that a 0 exit code means "things went fine" is pretty simple to defend - there's decades of convention to support this. Does it mean all apps will behave that way? Not necessarily, but I'm not sure you defend making the typical case more of a PITA because there may be broken applications in existence. :)
• Anonymous
  June 29, 2005
  Yeah, I'd say assume zero means success and prompt for non-zero. The onus is then on the third-party tool makers to ensure that if they really do fail, to at least delete the output file as well so as not to get the prompt from VSTS.
  
  I think when you're providing a new interface for applications to plug into, you're the one who decides on the contract, not the plugins. Given that you still want to work with existing apps, then the compromise you suggested is probably pretty OK.
• Anonymous
  July 04, 2005
  
  Abhinaba Basu on marking required fields in work items ⊕ and build types ⊕
  Adam Singer on the latest...
• Anonymous
  June 09, 2009
  PingBack from http://insomniacuresite.info/story.php?id=6024