I am very honored to be part of this amazing release of Visual Studio. I am enjoying it, I am sure you will too.

Hamed

1. A couple of people showed up a few minutes late; missing a part of the conversation, they would start asking some questions
2. If someone raised an impediment, others tried to help him and they would get into a detailed discussion
3. A developer described a work item finished the day before and the rest of the team started asking question about the outcome… sometimes 5 minute a person became normal
4. …

I actually tried establishing some rules about showing up late, basically “encouraging” people to be on time (I’m not going into the details, you can figure out what the best rules are based on your team’s culture). Then I brought a jingle bell to the standups, which would go off when a person’s two minute was up. That helped adjusting people’s status update pace, but did not improve the quality of the content; we were still hearing a lot of details.

So we made another change to refocus people on the relevant topics. Instead of going around the circle and ask everyone “what did you do, what are you going to do, any impediment”, we started going over stories on the Scrum board and asking whether there was an update since yesterday’s standup; what or why not. Going down through the list of stories, people would move the tasks on the board or raise hand if there were blocked. We then put a red sticky note on the board next to each story for correspondent blocking issues, but no one was allowed to get into the details at the standup. After the standup, interested parties would stay and discuss the issues. So here is a sample standup:

Me: “is there any update for story A”

Jon: “task A1 is now completed, task A2 and A3 are in progress, not blocked”

Me: “is there any update for story B”

Steve: “Task B3 is done, task B4 is blocked on network issue”

Me: “here is a red sticky for the network issue next to story B”

…

Steve, Jon and I then would discuss the details of the network issue after standup.

This might seem to be a very simple change, but it did actually help improve the efficiency of the standups significantly.

 --

Good day

Hamed

We appreciate your valuable feedback.

Hamed

One of the benefits that formal processes like Waterfall were supposed to deliver was to eliminate the chaos through the development of large projects, i.e. making the development more efficient.
However, I believe these models went too far and made the development “over efficient”. All the far ahead planning, allocating resources, and trying to stick with the schedule preempted all buffers in the development team’s capacity and left no room for reacting to the ever changing environment and dynamisms of the software development. The result is a reduction in the output or the value delivered to the customer. What agile practices have been emphasizing on is “productivity matters the most”.

Let me draw an analogy to make the matter more clear:

Consider a factory that manufactures a product. To do that, this factory utilizes four different machines M1, M2, M3, and M4. The raw material is the input to M1 and M2. The M1’s output goes into M3, and the output of M3 and M2 flows into M4, which outputs the final product. Also assume that the M3 is a bottleneck, that is, its capacity is less than that of the other machines.

To reach the maximum efficiency for each machine (based on the definition of efficiency from Mariam-Webster: effective operation as measured by a comparison of production with cost, as in energy, time, and money), we want them to be working 24/7. We invested a lot in these machines and the more idle they are, the less their individual efficiency will be. Now let’s look at the big picture: all machines are producing with their maximum capacity. Since M3 is a bottleneck, there is a pile of M1 parts in front of it ready to be processes. M4 also has a lot of M2 parts ready to go, but it is starving on M3 parts.

There are many major issues with this system, a few that comes to mind are:

1) It only can produce at the same pace as that of its bottleneck,

2) A lot of inventory is sitting to be processes (consider the risk of damage, cost of storage, etc.),

3) What if there is a required maintenance time for M3? Since we do not want to reduce the efficiency of other machines, we do not bother to pay the cost of repurposing another machine to produce M3 part, which is on the critical path of the product.

4) Any change in the product is very costly. Let’s say we do need to change the design of M1 part; all the excess M1 inventories are now wasted.

Now let’s redesign the system a bit: maybe M2 can be used to produce M3 parts as well (increasing the throughput), but there needs to be a setup cost to switch between parts each time. Also we slow down M1 to produce only as much as what machines that produce M3 parts can process (adding some idle time to M1). As you see, we decreased the efficiency of the individual machines to achieve higher productivity. We now produce more than before, deliver more to our customers and make more money.

A software development team is just like that factory: it produces products using limited resources with various capacities, only add the uncertainty of how long an activity would take and the fact that the 'machines' are much more sophisticated and multi-purpose. You can relate all the items I mentioned for the factory to a software development team. If the team shares the same goal and on the same mission I mentioned above, it needs to focus on its productivity, not efficiency! Any process or activity that the team goes through should get the team closer to its final goal. Conforming to this mindset is not easy though, it requires a culture change.

Once I heard from a project manager that “we pay these people, so better keep them busy all the time”! That is exactly the kind of thinking that could blow up a team’s productivity.

Final word: I believe the most efficient team is not the most productive one!

--

Good day to you

Hamed

Now let me tell you an story about an experience I had when I was in college. I worked on a project which was supposed to make me rich (you know, "student rich" not "business rich"!). There was a demand for a system that could transfer live videos through the phone line. Different industries were interested in this system because managers and top engineers wanted to monitor what was going on in the manufacturing plants, usually placed in remote areas with no high speed internet, fiber optics, or satellite equipment.

My friend and I were offered to develop the software to stream videos captured by different types of industrial cameras over the phone line. Basically, the software should encode the live video captured by the camera in station A, encode the video and send it through the phone line to another instance of the software sitting on the other end of the phone line, station B. The software in station B should decode the stream and play it. Everything looked straight forward until the customers introduced and insisted on a set of requirements:

1. The software should stream and play the video very fast (live, without a delay)

2. The quality of the video should not change during the transmission

3. The video should play smoothly in station B

We struggled a couple of months and reduced the delay between the time the video is fed to the software and when the video is played to six seconds. The quality changed during the transmission due to the encoding/decoding method we chose to reduce the delay. And also the video was playing with a few hiccups per minute because of the type of line (phone line) was used to transfer the stream.

When we demoed the software for the first time, one of the users who had never experienced video streaming before got excited and gave two thumbs up. He thought the delay was not really bad, the quality was acceptable, and the hiccups could be ignored. The other one, who regularly used high-speed internet video chatting, did not think the same way and he wanted all the requirements together...  I did not become a rich student after all!

OK, what we missed was defining proper performance expectations. I believe setting the correct set of performance goals, i.e. defining what our performance expectations are, is the half way through having a performant software. Sometimes, the goals are clear. For instance, the software that monitors a nuclear reactor should calculate different criteria, such as pressure and temperature, and alert the abnormal cases in less than one second (I made up the number just for the sake of the example), or consequences could be sever. This goal can be calculated scientifically and there is no compromises. However, in most cases unlike the nuclear reactor, customers just want the software to be fast, like our customers. There are some studies to be done and discussions to happen with the customer before setting the goals. 

From the user standpoint, the performance is mostly a perception; the user's impression of how fast the software responds. How a user perceives the performance actually depends on many factors including:

• The user's previous experiences and how the user used to do the task that the software does now: just compare our two users in the above example. One had used similar products (although for different purposes) so his expectation was different from that of the other user.

• What the software does and what end users do while the software is trying to respond to an event: users usually consider a software showing some sorts of visual indication on what it is doing, faster. The reason for that is the users do not like the "Not responding" state. Responding to the same event, a software with "not responding" state for five seconds looks slower than one that shows a progress bar for seven seconds. Do you think our customers would have considered the six second delay a bad performance if they were not sitting in front of the monitor and watching the live video?

• The scalability of  the software: imagine that our software streamed the video with no delay for two minutes and then it started streaming with a five second delay afterward. Users would still describe the software as "slow".

We definitely should translate the users' feelings into numbers to be able to measure them. For instance, the goal for the responsiveness of the software in the above project should have been set to something like "no more than six second delay".

The next question we need to answer is how the end users, and more importantly the customer, benefit from the software reaching a specific performance goal. When we asked our customers why there should be no delay on the video streaming, we did not hear any valid reason. There was no real time evaluation or analysis on the video in station B, thus, there was no business value on reaching a zero delay. So, you should always talk to your customer about the value achieving a performance goal delivers. Most of the customers do not do a cost-benefit analysis of reaching a certain performance level. In addition, to achieve a performance goal we might need to sacrifice other requirements. To reduce the streaming delay, we had to choose an encoding method that compressed the video to a level that impacted the quality, contradicting another objective. The decision maker here is again the customer, considering the business value each objective delivers.

Another important aspect to consider is the environment and hardware equipment the users are going to use to run the software. You probably can feel the performance difference running a software on different machines or OSs. Had we gotten a fiber optic infrastructure, we would have not struggled to reduce the delay. So there was a restriction on the equipment available to us. We also managed to improve the video hiccups by changing the video card. However, the customer could not afford buying the video card for all the users.

The last but not the least, there are some alternatives to compensate for a lower performance. A user who loves classic music could live with a two minute response time if the software plays a piece from Mozart for her (an exaggeration, but hopefully you get my point!). So open your mind and be innovative.

In summary, before thinking about how to develop a performant software, first, try to understand how the performance can impact the users'. Spend some time with your customer discussing the performance objectives and the values they deliver; help the customer to do a cost-benefit analysis. Also pay attention to contradicting objectives and restricting requirements.

Good day to you

Hamed

--

* C. U. Smith and L. G. Williams, Performance Solutions: A Practical Guide to Creating Responsive, Scalable Software, Boston, MA, Addison-Wesley, 2002

I will get back to you with a performance topic.

--

Hamed

Problem

To build something you need to have a sketch in mind, build it and test it to see how it works. That is almost the same story with developing software; it requires three activities regardless of the methodology being used: design, implementation, and testing. However, different methodologies utilize these activities in different ways. Some scrum developers still use "mini waterfalls" to develop user stories in each sprint and this mostly happens when a developer shifts from waterfall mindset to Agile. “Agile Development” tries to deliver shippable features in each sprint and be more focused on user stories and reacts to the changes. The sequential development process limits the agility of the development in a sense that a design should be at least partially completed to start implementation (don’t tell me you are one of those who jumps into implementation with absolutely no design at all!) and an end-to-end testing is not possible until a big piece of implementation is done, let’s say half way through the sprint. By testing I mean making sure that from the beginning to the end of the user story every piece of code works well and matches what user expects. Halfway through the sprint, what if the developer finds out that the implementation does not quite match the user story after running the end-to-end scenario for the first time? This is analogous to building different pieces of a car separately and then assembling them at once. The car might work well, but we could find out that it does not look good with the big bumpers or it is not comfortable because we did not realize that the dashboard is too big comparing to the whole interior space. This is the result of getting too deep into one piece before sketching the whole thing. We would have been able to see these issues if we’d had the “so-far” image of the car in mind during the building process or stepped back once in a while and took a look at the overall progress and the result. My main point is that with the current development process, the end-to-end scenario, which I call it “the flow” of the user story, does not come together till the near end of the implementation.

I believe agile development needs an interactive and flexible development practice that enables developers to focus on the essential development activities, customize the sequence of developing different pieces for each user story, change the design and implementation very easily, and most importantly start the end-to-end testing from the very beginning of the development. It is very crucial that the flow of the user story in the code matches what the users expect. It is very easy to get caught in the implementation details and lose the big picture.

In addition, we all have seen that documentation is not a priority for hardcore agile developers; however, I strongly believe that investing in “proper documentation” will pay off in the long run. Sometimes documentation is part of the deliverables and very clear how it should look like, but if it is not being delivered, the "proper documentation" should enable somebody new to the team or any team member after 2 years understands the system and be able to maintain it. But the truth is documents such are usually time-consuming to produce and get out of sync with the actual implementation over time.

I am proposing a practice in which you can address the above issues in your agile development environment. I am not trying to change how you design, implement, or test, rather just use a different order and introduce some steps in between.

I have tried this many times and gotten very good results so far. The practice could use more developers to adopt and lots of feedback (Thank you in advance J ). Since this development practice focuses on the end-to-end flow of the user story, I call it “Flow-Driven Development”. 

My Solution, the Flow-Driven Development

Overview:

Flow-driven development is a development process suitable for developing user stories in an agile environment such as Scrum. Developers integrate the design, implementation, and testing of a user story into one comprehensive phase, consisting three activities: Stubbing, Filling, and Walkthrough. Each of these activities tries to accomplish the goal of a waterfall phase, i.e. design, implementation, and testing, but they could be practiced in different orders to overcome the problems I described before. I need to mention that the details of executing these activities are decided based on the team and the project dynamics.

Also please note that this process relies heavily on code comments which satisfy a specific format presenting all necessary information for development activities (which could be standardized).

The practice in a high level:

Having a user story in hand and a good-enough design in mind, “stubbing” starts from high-level items to low-level ones (top-down approach). I want to stress on the fact that you do not need to have all the high-level objects in your system laid out before starting the stubbing; start from a few ones and other stubs will be added gradually as the process moves forward. Stubs are first created for classes. I suggest putting the following comments at the top of each class: 1) the concept in the system that the class represents (e.g. student), 2) responsibilities of the class (e.g. take courses), and 3) relations/interactions to other classes (e.g. has a teacher in each course). Other comments may be required occasionally. Then, stubs are created for class members to enable the class to accomplish its responsibilities and provide proper communication channels and interfaces to other classes and the outside world. I would provide each member at least with the following comments: 1) the description of data/functionality that the member caries (e.g. the number of credits the student has taken so far), 2) for functions, the description and the valid range of input/output and any exception/error the function may throw, and 3) who should/may use this member. We should be able to follow the flow of the user story from the beginning to the end by reading the comments. Be sure to have the code compile-able and runnable at all time; for instance, all methods and properties that need to return something have a return value hardcoded for now. The user story may not be complete now, but we should be able to walk through “a happy path” from the beginning to the end.

When you feel that you have laid down the skeleton of the user story, the filling starts. Keep in mind that we want to have a happy path of the end-to-end scenario stubbed at the beginning before getting lost into the implementation details. Each stub is being filled accomplishing what is described in comments. The order of the filing is totally up to you, but I like to start from simple stuff, so I fill a method that verifies an email address before the one that calculates a company’s revenue. The filling activity contains two steps: the implementation is described in natural language at the top of the stub and then converted to the code.

A unit test needs to be written if applicable (and yes, it is worth the time; if you care about the quality, write the unit test). You can write the unit test just after the filing or utilize the TDD, whichever you are more comfortable with.

The filled stub should be tested by walking through the different scenarios of the user story that the stub is involved in. During the walkthrough, the developer does what the user would do, but he also can see what is going on under the hood. The developer must refer to the user story to validate the flow has been implemented precisely. The developer can change the flow anytime if it does not “feel” or “look” right!

Developers can go back and forth between different activities if needed. For example, a class member stub maybe created during filling another member stub to provide required data/functionality. The return value of this new member can be hard-coded for that moment just to get us going and can be filled later. Also a walkthrough can be done before any filling just to make sure the code flows between different components correctly. In my opinion, this is a very good “design review” practice!

These three activities are repeated until all the stubs are filled completely and walkthroughs are successfully passed on all paths of the user story. Obviously unit tests are all green at this moment. Also, at the end, a tool can extract all the comments from the code to form a document. The better and more complete the comments, the better the document.

In summary, using mini waterfalls to implement user stories in sprints may not be the best option and has its own limitations. Since user stories are a key part of the agile development, we want to make sure that the delivered product does exactly what the stories are telling. This goal can be achieved more effectively if we always have the end-to-end scenario in mind. By integration the three phases of the traditional waterfall methodology and paying more attention to the flow of the user story we have a better chance of making our customers happy.

Good day to you.

Hamed

This is a common question which I came across many times in Forums. Here are simple and quick checks that you can do before you start scratching your head or jump to the Forums. I’ll be using Excel as the Office application, but the approach is applicable to others more or less.

1.       Make sure your add-in is installed correctly and it is enabled

Open registry editor and navigate to “HKEY_CURRENT_USER\Software\Microsoft\Office\Excel\Addins”. Here is how this registry key looks like after I installed MyExcelAddin:

The registry string we are interested in the most is the LoadBehavior. Whenever you see "3" (Loaded at Startup) as the value of this string you are good in this part and your add-in tries to load when the Excel starts. For more information about registry entries for application-level Office solutions see this article.

2.       Check out “COM Add-Ins” dialog in Excel

Now open Excel and see if your add-in loads successfully. If not, open Excel Options dialog (File->Options) and navigate to Add-Ins tab.

“Healthy add-ins” show up as “Active Application Add-ins”, however, if an add-in is disabled for whatever reason, it will appear under the “Inactive Application Add-ins”. You can select the “COM Add-ins” from the Manage dropdown at the bottom of this dialog and see the LoadBehavior of all COM add-ins. In normal cases, the add-in should be checked and its "Load Behavior" should be “Loaded at Startup”:

If the VSTO runtime encounters a problem loading the add-in, you’ll see this message instead.

“Load Behavior: Not Loaded. A runtime error occurred during the loading of the COM Add-in”.

3.       Find the problem

Set the "VSTO_SUPPRESSDISPLAYALERTS" environment variable to 0. Then go back to the registry key I mentioned in step 1 and set the LoadBehavior to 3. Now restart the Excel. You should be able to see a dialog describing what is happening:

It is also a good practice to enclose the code inside the add-in's startup event in a try-catch block and log all exceptions.

 4.       Hard disabled add-ins

If an add-in causes the Office application to crash (close unexpectedly), the add-in gets “hard disabled”. Please follow this instruction to re-enable the add-in. In addition, you might want to check the “HKEY_CURRENT_USER\Software\Microsoft\Office\12.0\Excel\Resiliency\DisabledItems” (replace 12.0 with 11.0 for Office 2003 and with 14.0 for Office 2010). Your add-in should not have an entry under this registry key.

More References:

• Debugging in Application-Level Projects
• TroubleShooting COM Add-In Load Failure
• VSTO blog
• VSTO forum

--

Good day to you!

Hamed

Well, at the first glance, this seemed to be a usual ClickOnce deployment which I had done before with different .NET applications many times: dealing with projects with more than one executables, I just needed to set the main project as “Startup Project”, include the second exe in the main project, and call the second exe from the main project using the Process.Start. I was never worried about the path to the second exe since it gets deployed next to the main one, so I always used relative path.

However, it is different when you try it with VSTO add-ins. The following code in my add-in startup could not find the exe file, despite the fact that “MySecondExe.exe” is deployed correctly:

Process.Start("MySecondExe.exe");

So, why the file could not be found? If you retrieved the value of the Current Directory in your add-in project and a WinForm project deployed through ClickOnce, the former returns something like “C:\Users\hamed\Documents”, and the latter returns the ClickOnce cache path. There you go! The base working directory is not what you would expect in add-in case. Easy to explain though, add-in is not an executable, it is a package of a manifest, assemblies, resources and other files such as data files. The manifest is parsed by the VSTOLoader, an AppDomain is created, and the add-in assembly gets loaded into the AppDomain. So we need to retrieve the assembly location, i.e. the ClickOnce cache path in which the add-in is deployed to, using the following statement:

string assemblyPath = Path.GetDirectoryName(Assembly.GetExecutingAssembly().Location);

You can then use the above path to locate your second exe file.

Good day!

Another requirement that mostly concerns corporates is to make an application or functionality, in this case a VSTO add-in, available for some users of a machine in an automated way. Basically, the idea is to have the add-in available based on the user credential. Of course the first solution that comes to mind is to install the add-in for all users and have the add-in decides who should have access to it. You should not be concerned about deployment if you are using this approach, however, you need to add the logic for authentication/authorization and decision making to grant/revoke access to users. I would say this is best if you want to filter functionalities in your add-in based on the user’s credential rather than making the whole add-in unavailable.

If you are using ClickOnce as your deployment method, I should say your hands are kind of tight for one-step installation, since the add-in gets installed in the ClickOnce cache which is a per-user storage. For instance, all add-ins installed under my credential lives under “C:\Users\hamed\AppData\Local\Apps\2.0”. That is why Thony could not get his add-in work just by copying the registry keys. In this case, you need to install the add-in for each user on your “have access” list. This does not seem easy in companies with great number of computers and users. However, do not let this solvable issue take all the great features of ClickOnce deployment away. An active directory group can be created and access to the installer on a shared server could be granted to the group. Therefore, each user on the list can install the add-in on her machine under her user account.

Using Windows Installer (MSI) gives you the option to choose where your add-in gets installed and, therefore, you can just have a batch file to copy the required registry keys to different users registry, having the add-in live on a shared path for those users. I am personally not in favor of this approach though, if I do not have specific reason to follow that. Customizing an MSI needs some degree of expertise and is sometimes error-prone, at least for novice users. The major drawback would be not having the auto-updating feature of the ClickOnce, which could be a pain point in big companies (read Enterprise level deployment by Kris for more information).

Always look for your environment limitations and needs comparing different approaches and find out which one gives you the best bang for your buck, especially in the long run.

Good luck.