Has it ever happen to you when working on a bug fix that after investing time on a fix you realized that the fix does not actually fixes the bug?

Or even worse, new bugs appear as the result of your change in the system under production.

This is similar to what happens when scientists assume a hypothesis to be true without actually running an experiment to confirm it. This something that no good scientist will ever do, however, it happens in the Software Engineering field more often than not.

If you compare the Scientific Method to the process followed to perform a bug fix, you will find a lot of similarities. I will go as far as to say that they are the same process.

Let me elaborate, the Scientific Method defines the following mayor tasks:

1. Define the question

2. Gather information and resources (observe)

3. Form hypothesis

4. Perform experiment and collect data

5. Analyze data

6. Interpret data and draw conclusions that serve as a starting point for new hypothesis

7. Publish results

8. Retest (frequently done by other scientists)

Let see how they apply to Software development and in particular to the process of “bug fixing”.

1: Define the question:

For software development the question is simple: “Is the Software under development done-done?”

By done-done, I mean: Does it complies with user requirements? Does it have bugs? Is it shippable to customers?

Most of the time (if not always), the answer is no. Software will always have bugs and new feature requests from customers.

2. Gather information and resources (observe)

This step is formalized in software development by testing and filling bugs against the software in development. A good bug is one that actually contains the right information and no more. Adding more data will just create noise and make the bug harder to read and understand. This will serve the triage team to decide if team wants to move to step 3.

3. Form hypothesis

This is the step in software development when a developer gets assigned a bug. The next step for the developer is to “guess” what’s going on. I say “guess” because by just reading the bug, most developers will not know for a fact (unless they were the ones that file the bug) what is going on.

The developer may use the debugger to understand and to gather more data about the bug; this could already be done during step 2.

A failing unit test is a great way to formulate a hypothesis to describe the bug under fix. If you find yourself in a position where it is impossible to write a unit test to express the bug under fixing, then you need to step back and invest into building the right infrastructure to be able to formulate hypothesis against your system under development. This is fancy way of saying that you need to build infrastructure that lets you build unit tests.

Another sign of problem in the process is if extensive debugging sessions (lasting more than 10 minutes) are required to formulate a hypothesis about the problem. But this is an entire subject for a different blog post.

Let me emphasize the point about investing in test infrastructure. If you compare the kind of investment made by the scientific community on “experimentation infrastructure”, we in the software development community fall short by a long shot. When did a software project had the need to build a particle collider or to put a huge telescope in orbit just to be able to “test” an hypothesis?  

4-5. Perform experiment and collect data and analyze data

These steps are together because unlike scientist, developers have it easy. They get to always use computers to run unit tests and integrations tests and most tools collect enough data about the system under tests.

I have to note than just like scientists, sometimes developers also need to design and implement experiments. Typical examples of this are:

a.       Design a stress test framework.

b.      Implement your own testing framework from scratch, like XUnit or NUnit if your development system lacks one.

6. Interpret data and draw conclusions that serve as a starting point for new hypothesis

This is the step where important questions need to be asked, like: Is there more cases that our hypothesis is not covering? Why this bug was not discovered earlier? Is this a regression?

Do we have invalid hypothesis (unit tests that actually do not test the system) in our system?

Once the developer confirms that in fact the unit test is valid, then she must proceed to perform the actual fix for the bug, which is equivalent in science as to formulate a new theory or model. In the case of software development we need to find the “the simpler solution that could possibly work but no simpler”.

Scientist and developer, we both love when we find a solution that is: simple, small and complete. And also, both groups of professionals are very suspicious of the solution if it is complex and large. Some scientists will go as far as say that these kinds of solutions are really an intermediate step into the final and elegant solution. I tend to agree with that.

But developers are also engineers and we need to balance cost and time to be able to ship. Shipping is something that does not concern scientist (but there have been some famous races between scientists to see who is able to publish first).

7. Publish results

This is simple; just hit the button to do the submission in your source control system.

8. Retest.

In the scientific community a new theory, model or result is published with the expressed intent to be re-validated by other members of the community. It is enough that a single person in the community finds the result “invalid” to restart the process back to step 2. Software development is the same, once a bug is declared fixed; another member of team (not the same developer) must retest the bug and validate the fix.

In software development, some part of this step is automated if you have a gated check in system.

Developers hate when a bug fix is re-opened, that means the formulated hypothesis was incorrect, and the unit test created (if any) is not testing the right aspect of the system under development.

If the bug is actually fixed then the process goes back to step 1. Which is to ask: Are we done-done?

Conclusion

I focus the comparison with the process of bug fixing, but the comparison applies to the process of developing user stories. And with that almost the entire software development process is covered.

I would to make emphasis on step 3, myself being a software developer; when someone asks you about why a bug happens, do not guess, instead, formulate a hypothesis and run an experiment, then answer the question. That’s what a sane scientist will do. Even worse, do not check in code “hoping” that it will fix a bug. That’s like a scientist publishing a new theory without experimentation. Any scientist that does that not only will be ridiculed by her peers, she can forget about making a career as a scientist.

In short, engineers that do software development for a living should act more like scientist, because like it or not we are doing the same job scientists do.

1. It did not work from a 64bit PowerShell

2. For some reason Get-Batchfile did not work for me, I had to modify to call "echo" explicitly.

I also added a helper VsInstallDir and a alias for devenv.

 Enjoy,

 Oscar

function global:VsInstallDir($version="10.0")

{

  $VSKey = $null

  if (test-path HKLM:SOFTWARE\Wow6432Node\Microsoft\VisualStudio\$version)

  {

    $VSKey = get-itemproperty HKLM:SOFTWARE\Wow6432Node\Microsoft\VisualStudio\$version

  }

  else

  {

    if (test-path HKLM:SOFTWARE\Microsoft\VisualStudio\$version)

    {

        $VSKey = get-itemproperty HKLM:SOFTWARE\Microsoft\VisualStudio\$version

    }

  }

  if ($VSKey -eq $null)

  {

      throw "Visual Studio not installed"

  }

  [System.IO.Path]::GetDirectoryName($VsKey.InstallDir)

}

function Get-Batchfile ($file) {

    $cmd = "echo off & `"$file`" & set"

    cmd /c $cmd | Foreach-Object {

        $p, $v = $_.split('=')

        Set-Item -path env:$p -value $v

    }

}

function global:VsVars32($version="10.0")

{

  $VsInstallPath = VsInstallDir($version)

  $VsToolsDir = [System.IO.Path]::GetDirectoryName($VsInstallPath)

  $VsToolsDir = [System.IO.Path]::Combine($VsToolsDir, "Tools")

  $BatchFile = [System.IO.Path]::Combine($VsToolsDir, "vsvars32.bat")

  Get-Batchfile $BatchFile

  [System.Console]::Title = "Visual Studio shell"

}

set-alias devenv ((VsInstallDir)+"\devenv.exe") -scope global

First a disclaimer: running these steps will lower the security settings of your PC, use with caution. 

Here there are:

In your encoding server (mine is called tv-server):

Enable-PSRemoting  -force # Enables remoting
set-item WSMan:\localhost\Client\TrustedHosts * -force # I am not running a domain controller, so my server needs to trust everybody
set-item wsman:localhost\Shell\MaxMemoryPerShellMB 1024 -force # The default allowed memory is very low, we need to bump it so that Convert-Media have room to work.
Enable-WSManCredSSP -Role Server -force # I need the server to have access to my Windows Home Server, so it needs to autenticate using credential delegation

In your client:

function AllowDelegation($policyName)
{
    $keyPath = "HKLM:\SOFTWARE\Policies\Microsoft\Windows\CredentialsDelegation\"+$policyName
    if (test-path $keyPath)
    {
        rmdir $keyPath -force -rec
    }
    $key = mkdir $keyPath
    $key.SetValue("1","WSMAN/*");
}
AllowDelegation "AllowDefaultCredentials"

AllowDelegation "AllowFreshCredentials"

Enable-WSManCredSSP -Role Client -DelegateComputer * -force

In order to remote into your server do:

Enter-PSsession <ServerName> –cred <UserName>

I have used “*” (star) in two places, that’s too broad, if you are already using a DNS suffix I would recommend to use it and change “*” to “*.mydomain.com”.

In that case you need to refer to your server as:

Enter-PSsession <ServerName>.mydomain.com –cred <UserName>

Once inside, load the Encoder module as usual and execute Convert-Media.

You could Batch background encoding jobs using the instructions from the PowerShell team.

Enjoy,

Visdual Studio 2010 Beta is now available here.

Please start pounding it on it and let us know what you think.

The author explains why some unconsiuos cultures can come up with better home designs than the developed (or self-consiuos) cultures. It also explains why a desing can only be created using test or "proofs".

Form there you can extrapolate to software development and see with a complete new perpective how the agile process approach the process of "Design".

In my personal case, It allow me to see why principles like YAGNI and KISS are fundamental to the process of design.

Even if you are not into agile, I definitely recommend this book, its a short read.

- New UML Designers 

- Architecture Explorer

- Layer Designer

Please download a give us feedback at Dev10 CTP Feedback page

If you have questions about the UML designers use the forums, we will be watching.

I have one week using it, I am not going back to cmd.exe ever!! 

Here is a great replacement for vsvars32.bat made by Brad Wilson.

See you there.

Joining Microsoft has been a huge change for me, both professionally and personally (I relocated from Costa Rica) and it's been an incredible busy year, so forgive me if I haven't answer your CAB and/or GAT questions (I know there are a couple here and there). Meanwhile I move the CAB and GAT articles from my previous employer web site to the msdn blog site.

Regards,

Oscar Calvo