Can we apply concepts from operations to software development????????

This blog is based on a group assignment we (the Nutmeg group) had in the operations class at NYU Stern School of Business. I thought it would be of interest, so I just took it and published it here....

Industry Background

The software industry has a history of over promising, cost overruns, delay of releases, and poor quality at release only to improve after multiple revisions. These problems are very well captured in the CHAOS report which is a yearly report on the state of the software industry by the Standish Group[1]. The most recent study of the Standish Group shows today only approximately 30% of all software projects can be called a success. Approximately another 50% of projects are challenged, which means they run over budget and deliver fewer features than originally promised. The remaining 20% of projects are considered failures and are cancelled before they deliver any value. Looking back into previous reports, we realize that these numbers have not changed much over the last decade. While we can argue the exact numbers presented, these results are a clear indication that there is some fundamental problem with the way we approach software development. Research over the last 10 years tries to identify the main reasons for this relatively high failure rate of software development – especially if we compare these numbers to other engineering disciplines. The result of the research so far is twofold:

1. Software developers could experience an improvement from the “waterfall” style project management to more agile, iterative development processes that are broken up into distinct phases to avoid the big bang integration issues that were very common in the late 1990s. For example, the NASA Goddard Space Flight Center[2] defines the following phased approach for their software development:

Figure 1: NASA Software Systems Development Process

2. Software developers could also observe an evolution of programming languages, tools, and techniques. One of the main reasons for the difficulties in the software industry is the lack of standardized, reusable components. What that means is that instead of reusing components that are well defined with known behavior (comparable to building elements used in civil engineering with known resistance to forces) we find a culture of “not invented here” syndrome by many companies and product teams. The result is a wide variety of non-standard components with slightly different behaviors and overall low quality. Furthermore, in the search for more systematic reuse, the industry tried to find the right level of granularity for reuse. Therefore, the software industry went through the evolution from functional programming to object oriented programming and then to Component Oriented programming and in recent years to Service Oriented Programming paradigms throughout the last 20 years. This progress resulted in slight improvement of the situation and enabled better manageability of the increasing size and complexity of software systems, but it failed to better the situation dramatically.

How can Operations Management be translated into the software industry?

While the concepts of operations can easily be applied to manufacturing, its applicability is not straightforward to the service industry, in particular to software. Let’s first consider the inputs for Software Development, which usually are requirements. These requirements are mostly intangible business problems. Therefore the lead-time for production cannot be specified in terms of acquiring resources and or parts. Because the requirements are not tangible they are usually abstractions of the problem that need to be solved in the business domain. Unlike manufacturing, over a period of time, the requirements for software systems tend to change much more frequently than the technology and/or marketplace changes. In fact, many times the customer does not even realize what he/she really wants in the early stages of the project (this is one of the main reasons 50% of Software projects are challenged and 20% fail). Furthermore, in manufacturing we have the ability to measure productivity against benchmarks and we can usually measure quality as easily. In contrast, software development cannot be measured with the same ease. One of the reasons is that the production of copies is as easy as copying a file whereas creating the actual file the first time is – up to today – more of an art-form. Even worse, scientific studies show that productivity fluctuates 10 fold[3] between individual contributors to a system. Measuring the productivity and quality of a software system is extremely complex because of the magnitude of possible variations (e.g. for Windows Vista there are 70,000 drivers certified to be compatible). Therefore, measuring progress and throughput of software development is still an area for research and as Bill Gates stated “Measuring the progress of software development progress by lines of code, is like measuring progress of aircraft manufacturing by weight”.

Applying the Concepts of operation to the Software Industry

There are two potential ways how the concepts introduced in class could be applied to projects in the software industry:

1. On a project-level basis: In this approach, we base the measurements of throughput on the number of projects released into production. We can split the development process down into the different phases of development as shown in Figure 1. Based on the different phases, we can identify bottlenecks using the MRI analysis, as introduced in class. The problem with this approach is that software projects are not as uniform as the production of many similar cars, for example. By its nature, copying software is very efficient but the development is very unique for each project and product. Software is therefore more considered a one-off craft rather than product line type manufacturing having standardized components. There are initiatives such as “Software Factories”[4]

2. On a feature basis: In this approach, we measure throughput based on features. This seems to be a more appropriate measure for most of the software development projects. The problem here is that it is very difficult to map each requirement through the entire process until the end product. The reason is that – unlike in manufacturing requirements – software requirements are often defined very loosely and, more often than not, these requirements change over time. While we can measure function points in the implementation of a manufacturing project and its mapping back to the requirements, the design, and the testing, this is much more difficult in a software project. Part of the problem is cross cutting concerns such as security, maintenance, extensibility, etc. These aspects of a system cannot be mapped one-to-one backwards from implementation to requirements since it is a many-to-many relationship. This returns us to the earlier point that in manufacturing, performance can be easily predicted because of the use of standard components with well-defined performance characteristics; in software, we use one-off components that are not standardized and therefore their characteristic data are not known upfront.

General Rules For Successful Software Development

Successful software development projects follow a set of principles to arrive at a positive outcome:

1) Outstanding users who understand the intended software use and overall business environment are recruited as champions of the software. These users serve as a team to gather a full set of requirements for the new software or software upgrade. It is important that these users have a good understanding of technology as well.

2) Good specifications are created for the intended software via a detailed preparation of “Use Cases” specifying exactly how the software is to behave. These use cases can be quite lengthy and force questions to asked about critical junctures in the software actions. For a large project, it is important to have the users sign off on these use cases.

3) Use rapid prototyping tools to generate screens and screen flows – even if these screens are throw-aways – and iterate this prototype with the users to get their feedback. The best feedback from users often comes when they can visualize the system via a tangible set of screens.

4) If the system involves a relational database, the full “Entity Relational Diagram” should be created in a design tool such as ERWin. This allows a detailed mapping to be created between the screen content and how the data is housed in the database.

5) After this lengthy process of specification gathering in many forms, the actual system can be created. If possible, the prototype can be evolved into the production system. As such, the database design is accomplished. With the screen flow established and the behavior specified in detail in the use cases, the variables that can cause project failure have been minimized.

6) Following the software development process, detailed testing is aided via the use cases. Testing is in two phases as the new system is first tested internal to the development team and then second is tested with the users via “User Acceptance Testing” (UAT). In the UAT process, minor changes can be effected to tailor the system to the users' environment. Major changes are avoided through the screen prototyping and use cases.

---

[1] https://www.standishgroup.com/

[2] https://software.gsfc.nasa.gov/isdpaindx1.cfm?selcat=Product%20Development

[3] https://www.amazon.com/Peopleware-Productive-Projects-Teams-Second/dp/0932633439/ref=pd_bbs_sr_1?ie=UTF8&s=books&qid=1216176210&sr=1-1

[4] https://www.amazon.com/Practical-Software-Factories-Books-Professionals/dp/159059665X/ref=sr_1_2?ie=UTF8&s=books&qid=1216175970&sr=8-2