Ten Year Agile Retrospective: How We Can Improve In The Next Ten Years
Jeff Sutherland invented Scrum in 1993 and worked with Ken Schwaber to formalize Scrum at OOPSLA'95. Together, they extended and enhanced Scrum at many software companies and helped write the Agile Manifesto. Jeff's blog reviews the origins and best practices of Scrum at http://scrum.jeffsutherland.com.
June 2011
Ten years after the publication of the Agile Manifesto, Jeff Sutherland describes the successes of Agile and pinpoints four key success factors for the next ten years.
Applies To
Agile, Scrum, software development, project management, Team Foundation Server
Introduction
Key Success Factor #1: Demand Technical Excellence
Key Success Factor #2: Promote Individual Change and Lead Organizational Change
Key Success Factor #3: Organize Knowledge and Improve Education
Key Success Factor #4: Maximize Value Creation Across the Entire Process
Conclusion
Ten years after the Agile Manifesto was published, some of the original signatories met with a larger group of Agile thought leaders met at Snowbird, Utah, to do a retrospective on 10 years of Agile software development. They celebrated the success of the Agile approach to product development and reviewed the key impediments to building on that success. And they came to unanimous agreement on four key success factors for the next 10 years:
Demand technical excellence.
Promote individual change and lead organizational change.
Organize knowledge and improve education.
Maximize value creation across the entire process.
In this article, I describe the key success factors as determined by the 10 Year Agile Retrospective, and then I list the highest priority problems that keep organizations from acting on those success factors.
Key Success Factor #1: Demand Technical Excellence
Teams that deploy applications in short increments and get rapid feedback from end users have largely driven the explosion of the Internet and the proliferation of applications on smartphones. This is formalized in Agile practice by developing products in short time boxes, which are called sprints and last no more than a month and most often two weeks. We framed this issue in the Agile Manifesto by saying that “we value working software over comprehensive documentation.”
The 10 Year Agile Retrospective concluded that the majority of Agile teams are still having difficulty testing their work within sprints because neither the management, nor the business, nor the customers, nor the development teams demand technical excellence.
Impediment #1: Lack of READY Product Backlog
The Product Owner is responsible for working with stakeholders and customers to develop a product backlog that is broken down into small pieces, clear to the developers, immediately actionable, estimated in points by the team that will implement it, and testable (i.e., acceptance tests are clearly defined that determine whether a backlog item is done). A strong definition of done that is continuously improved is a hallmark of a high performing Agile team.
Most development teams do not have a good product backlog. I regularly query Scrum practitioners in the Americas, Europe, and Asia on what type of backlog they have. Over 80% have user stories for their product backlog, but less than 10% find their current user stories acceptable.
Systematic, a CMMI Level 5 company with extensive published data, has shown that their teams consistently double their velocity when product backlog is in a high ready state because the team must do only half the work[1]. So poor product backlog will typically reduce team performance by at least 50%. Complicating the problem, customers rarely or never use 65% of features that are created on average worldwide[2]. Poor product backlogs will have more unused features, as well as poor quality in the features that are used. If you develop less than half as fast as you could and 65% of what you build is waste, your performance is only 17.5% of what it could be.
Impediment #2: Lack of DONE Product at the End of a Sprint
You calculate team velocity by adding up the points for the product backlog items that are done at the end of each sprint. The product owner uses velocity to build release plans and a product roadmap. Teams use velocity to assess performance improvement. And management and boards of directors use velocity to assess the accuracy of product development plans and impact on revenue.
Lack of ready product backlog makes it impossible to get software done at the end of a sprint. Incomplete software is often the result of poor engineering practices, such as the following:
lack of testing during a sprint
poor configuration management
failure to implement continuous integration and automated testing
minimal or no pair programming or code reviews
All this leads to software that is not done. This means the team has no clear velocity, and the product owner cannot predict release dates or build a product roadmap to communicate with customers. The latest Scrum Guide[3] specifies that the product owner has a Release Burndown Chart that includes any work that is not done. For example, if 60 points burn down and there are still 40 points of bug fixing, integration testing, system testing, security testing, and documentation before release, the release burndown must burn up 40 points. Making this visible in the chart will clarify to the team and to management that for every six sprints completed, four sprints of undone work will remain before software can be put into production.
At the same time, if software is not done, testing occurs in later sprints. When coaching Palm, Inc. in 2006, we found that one hour of testing at code complete turned into 24 hours of testing effort three weeks later. The impact was that it would take two years to finish features that could be done in a month. Although this might be an extreme case, at Openview Venture Partners, we use Scrum in the venture teams and in portfolio companies[4]. We have never seen a company that did not take twice as long to deliver software when testing occurred in a sprint later than the sprint in which code was completed.
For these and many other reasons, Agile thought leaders agree that demanding technical excellence is the top priority for Agile management, Agile customers, Agile developers, and Agile stakeholders for the next 10 years. We know from Systematic data[1, 5, 6] and data from many other companies[7] that this change will at least double productivity in most teams, and it will quadruple productivity in high-performing teams.
Key Success Factor #2: Promote individual change and lead organizational change
In addition to technical excellence, Agile adoption requires rapid response to changing requirements. This was the fourth principle of the Agile Manifesto – “respond to change over following a plan.” However, individuals adapting to change is not enough. Organizations must be structured for Agile response. Failure to remove impediments that block progress destroys existing high-performing teams and prevents the formation of new high-performing teams.
Agility requires a significant mindset change: from focusing on a big upfront plan, to focusing on delivering the maximum value to customers, who are always changing their minds for good reasons. For example, customers get a better understanding of their business or their market changes, and they need software to adapt to those changes. Over 65% of requirements change during development for the average project worldwide, and the pace of change is increasing[2].
Impediment #1: Failure to See Impediments
The Scrum Daily Meeting is designed to surface blocks or impediments so that the team can remove them. Often individual team members are trying to work in isolation on their individual stories. A large number of items are open, almost nothing is done, sprint failure is almost certain, and team members will say they have no impediments working on their own little piece. Thus the team is actually a group of isolated individuals and not working as a team. They fail to see that they are their own biggest impediment. They need to individually change their behavior to work as a team.
Impediment #2: Tolerating Defects
A second major pattern of failure is a team not seeing any impediments despite having many open defects that are not quickly repaired. Industry data shows that fixing bugs on the same day as they are discovered will double the velocity of a team[6].
In order for Agile teams to succeed, Agile individuals must train, motivate, and lead the organization through a major change process. Failure or inability to lead this organizational change is the second most important issue that blocks Agile progress in enterprises today.
Scrum is a continuous process improvement approach to identifying and removing impediments to performance on a daily basis. Teams need to identify impediments in daily meetings and retrospectives and remove them quickly. Often 80% of the impediments require management help to remove. Management needs to understand Agile development and participate fully in its success. That means management must change along with the teams and lead the company forward. The widespread failure to do this in the face of increasing Agile adoption is of great concern to Agile leadership. Thus Agile leaders must lead organizational change.
Impediment #3: ScrumMaster Is Not An Agent of Change
A recent survey of 91 countries showed that 75% of Agile development worldwide is based on Scrum[8]. The ScrumMaster owns the process, is responsible for team performance, and must educate everyone involved on how to continuously improve by removing impediments. When the ScrumMaster finds an impediment that is outside the team but hinders team performance, the ScrumMaster is responsible for educating, training, and motivating people to take action. A good ScrumMaster is a catalyst for change in the organization. This responsibility is embodied in the work of Takeuchi and Nonaka that inspired Scrum[9].
In order to lead organizational change, we must provide better training and motivation for people at all levels in the organization. This means that the number three priority for Agile leadership in the next 10 years is to organize knowledge and improve education.
Key Success Factor #3: Organize Knowledge and Improve Education
Most managers and many developers are unaware of the large body of knowledge regarding teams and productivity. Scrum was designed to help teams incorporate best practices that evolved over many decades. Many of these best practices were formalized by the patterns movement and directly influenced the creation of Scrum and Extreme Programming[10]. Here are a few basic principles that are not well understood in the development community.
Impediment #1: Software Development is Inherently Unpredictable
Few people are aware of Ziv’s Law, that software development is unpredictable[11]. The failure rate on projects worldwide is over 65%, largely due to lack of understanding of this problem and the proper approach to deal with it. Traditional project management creates a plan that is expected to be delivered on time, within budget, and with predefined features. Yet, in the average project, over 65% of features change during development. Process engineering experts Ogunnaike and Ray[12] informed the co-creators of Scrum that using a predictive control system (waterfall) to control an empirical process (software development) was the origin of almost all chemical plant explosions and was at the root of most software project failures. These experts advised us to make sure that Scrum was an empirical control system based on inspect and adapt feedback loops.
Impediment #2: Users Do Not Know What They Want Until They See Working Software
Traditional project management assumes that users know what they want before software is built. As a result, over 65% of features built are either rarely or never used by the customers. This problem was formalized as “Humphrey’s Law” [13], yet it is systematically ignored in university and industry training of managers and project leaders.
Impediment #3: The Structure of the Organization Will Be Embedded in the Code
A third example of a major problem that is not generally understood is “Conway’s Law:” the structure of the organization will be reflected in the code[10]. A traditional hierarchical organizational structure will negatively affect object-oriented design, resulting in brittle code, bad architecture, poor maintainability and adaptability, along with excessive costs and high failure rates. Agile organizational patterns are designed to provide an organizational structure that supports good object design, which includes flexibility, adaptability, self-organization, reflection, and effective communication throughout the system via message passing, inheritance, and so on. Outmoded organizational structures fail to do this and produce bad code.
These are a few of the many principles that any organization that produces software must understand well. Fortunately, these principles are all encapsulated in good Agile practices. However, getting management support for these practices is a major impediment in most companies due to lack of ongoing education in fundamental principles for everyone from university students to boardroom directors.
Key Success Factor #4: Maximize value creation across the entire process
Agile practices can easily double or triple the productivity of a software development team if the product backlog is ready and the software is done at the end of a sprint. The bottleneck in most companies today is testing. Too often, software is not fully tested inside a sprint. That delay at least doubles the required testing later and, in some cases, requires 24 times as much testing. Fixing this problem will at least double a team's productivity.
Once a team starts going twice as fast, the product owner or product owner team must produce twice as much product backlog information. If there are poor user stories to begin with, they will be even worse when a development teams asks for twice as many of them.
Impediment #1: Lack of Agility in Operations and Infrastructure
As soon as talent and resources are applied to improve the product backlog, the flow of software to production will as least double. In some cases, the flow will be 5-10 times higher. This increase can cripple production, and any problems with development operations and infrastructure must be fixed. For example, a recent company transformation doubled the velocity of 27 Scrum teams. The lack of continuous integration produced twice as many bugs that could not be fixed within the sprint in which they were created. This delay generated several months of bug fixing after code complete and before product release. Even with this resultant drag on productivity, the other improvements in Agile approaches within the company enabled it to cut its development and deployment time from 19 months to 9 months with more features[14].
Impediment #2: Lack of Agility in Management, Sales, Marketing, and Product Management
At the front end of the process, business goals, strategies, and objectives are often not clear. This results in a flat or decaying revenue stream even when production of software doubles.
For this reason, everyone in an organization needs to be educated and trained on how to optimize performance across the whole value stream. Agile individuals need to lead this educational process by improving their ability to organize knowledge and train the whole organization. The few organizations that have trained everyone in Scrum all at once have always doubled, sometimes quadrupled, and occasionally gained market domination in one year.
The Bottom Line
It is essential that Agile teams improve their practices to resolve the largest problem facing the worldwide Agile community –technical excellence. Getting product backlog ready requires professional product managers that understand user needs and team capabilities with a passion for delivering excellence. Getting product backlog done in a sprint requires prioritizing work, continuously integrating work in progress, and intolerance of defects. Demanding technical excellence is the top priority for the next ten years.
In order to improve, Agile teams must promote individual change in development practices and, at the same time, lead organizational changes that enable greater output of software to the market.
To move the organization forward, Agile developers need to organize their knowledge and develop data-driven communication that motivates the organization to change.
At the end of the day, everyone in the organization needs to focus on the value stream, from the initial concept to revenue generation. Agile developers understand this problem. They need to communicate solutions and lead organizational change. Only then will the principles and values articulated in the Agile Manifesto be fully realized.
C. Jakobsen and J. Sutherland, "Scrum and CMMI – Going from Good to Great: are you ready-ready to be done-done?," in Agile 2009, Chicago, 2009.
J. Johnson, "Standish Group Study Report," presented at the XP2002, Sardinia, 2002.
K. Schwaber and J. Sutherland, Scrum Guide: Scrum.org and Scrum, Inc., 2011.
Sutherland and I. Altman, "Take No Prisoners: How a Venture Capital Group Does Scrum," in Agile 2009, Chicago, 2009.
C. R. Jakobsen and K. A. Johnson, "Mature Agile with a Twist of CMMI," in Agile 2008, Toronto, 2008.
J. Sutherland, C. Jakobsen, and K. Johnson, "Scrum and CMMI Level 5: A Magic Potion for Code Warriors!," in Agile 2007, Washington, D.C., 2007.
G. Benefield, "Rolling Out Agile at a Large Enterprise," in HICSS'41, Hawaii International Conference on Software Systems, Big Island, Hawaii, 2008.
VersionOne, "5th Annual Survey: 2010 - The State of Agile Development," Version One2011.
H. Takeuchi and I. Nonaka, "The New New Product Development Game," Harvard Business Review, 1986.
J. Coplien and N. Narrison, Organizational Patterns of Agile Software Development: Prentice Hall, 2004.
H. Ziv and D. Richardson, "The Uncertainty Principle in Software Engineering," in submitted to Proceedings of the 19th International Conference on Software Engineering (ICSE'97), 1997.
B. A. Ogunnaike and W. H. Ray, Process Dynamics, Modeling, and Control: Oxford University Press, 1994.
W. Humphrey. (2009, The Watts New? Collection: Columns by the SEI's Watts Humphrey. news@sei.
J. Sutherland and R. Frohman, "Hitting the Wall: What to Do When High Performing Scrum Teams Overwhelm Operations and Infrastructure," in Hawaii International Conference on Software Systems, Kauai, Hawaii, 2011.