Agile Software Development Methods

Gary Mrenak, Territa Poston, Jeffrey Schein

Contents

	Origins and Motivations
	Non-Agile or “Plan-Driven” Methods Characteristics of Plan-Driven Methods
		Plans/Documentation Milestones
	Criticisms
		Efficacy Documentation Handling Changes
	Agile Methods
	The Agile Methods Manifesto Current Variations of Agile Methods
		Extreme Programming Crystal Methods Dynamic Systems Development Methodology Feature Driven Development Lean Development Scrum
	Applications
		Financial Large Project Clinical Maintenance Embedded Safety-Critical
	Criticisms
	The Future of Agile Methods
	A Middle Ground
		Developers Customers Requirements Architecture Refactoring Size Balancing the Risks
	The Unanswered Questions
		Will Agile Methods Work For Average Developers? How Good Are Agile Methods?
	References

 

Origins and Motivations

Non-Agile or “Plan-Driven” Methods

Agile Methods are a reaction to traditional methods of Software Development, the so-called “Plan-Driven” methods. One of the earliest and best examples of “Plan-Driven” methods is the “Waterfall” method. Although there are different implementations of this method (for example DoD-Std-2167), all are similar to the one that [Schach, 2002] describes with a requirements phase, a specification phase, a design phase, an implementation phase, and a maintenance phase.

Characteristics of Plan-Driven Methods

Plan-Driven methods have characteristics that starkly contrast with those of Agile methods, and they revolve around documentation and development milestones.

Plans/Documentation

“Plan-Driven” methods, as the name implies, usually involve formal and continuous planning and depend on documentation to express and communicate the resulting concepts and decisions. [Schach, 2002] describes the Waterfall method as a “documentation driven” method where “..no phase is complete until the documentation for that phase has been completed and the products for that phase have been approved by the SQA [Software Quality Assurance] group.” The requirements and analysis phases of the method support the planning effort for the project and produce a specification document and a software project management plan. Once approved by the client, the specification document becomes the basis for the design phase which produces architectural and detailed design specifications.

A concrete example of a Plan-Driven method with a strong emphasis on plans/documentation is Dod-Std-2167, Defense System Software Development. Although now formally obsolete and long superceded by less prescriptive methods, the purpose of this standard was to “.. establish requirements to be applied during the acquisition, development, or support of software systems” [BMP 2002]. It described a comprehensive document-driven process that specified the required activities of a substantial software development process:

              •            Systems requirements analysis/design

              •            Software requirements analysis/design

              •            Preliminary design

              •            Detailed design

              •            Coding and CSU (computer software unit) training

              •            CSC (computer software component) integration and testing

              •            CSCI (computer system configuration item) testing

              •            System integration and testing

To support this process, developers were required to “..use systematic and well documented software development methods..” while performing each of these activities; they were required to “..document and implement plans for the installation, configuration control, and maintenance of each item of ..” the software development environment that they would establish; they were required to produce a Software Development Plan, Software Requirements Specification, and an Interface Requirements Specification; they were required to document and implement design and coding standards; they were required to document the development of CSUs, CSCs, and CSCIs in Software Development Files, one for each CSU, CSC, and CSCI, as well as document and implement procedures for establishing and maintaining SDFs, and they were required to produce a Software Product Specification for each CSCI. And there were similar requirements for Formal Qualification Testing, Software Product Evaluations, Software Configuration Management, and Transitioning to Software Support.

To underline the importance of documentation to the process, DoD-Std-2167 specified Data Item Definitions, templates prescribing the contents of each individual required document. Table 1 is a list of the documents specified by DoD-Std-2167 and their associated Data-Item-Descriptions.

This long list of documents and document content definitions is impressive. It is easy to see why [Schach, 2002] describes the software development process prescribed by DoD-Std-2167 as “document driven.”

Milestones

“Plan-Driven” methods use milestones (decision points) to both synchronize client and developer understandings and establish a point of common departure for the next phases and iterations of development. In DoD-Std-2167, formal milestones were prescribed by another standard, Mil-Std-1521,Technical Reviews and Audits for Systems, Equipments, and Computer Software, and were tied to the completion of seven of the eight prescribed major phases of the process: to formally complete the System requirements analysis/design phase, developers were required to support both a Systems Requirements Review and a Systems Design Review. To formally complete the Software requirements analysis/design phase, one or more Software Specification Reviews. For the Preliminary design phase, one or more Preliminary Design Reviews. For the Detailed design phase, one or more Critical Design Reviews. For CSC integration and testing, one or more Test Readiness Reviews. For CSCI testing and systems integration and testing, one or more Functional Configuration Audits and Physical Configuration Audits.

Documents are the currency of these milestones. That is, the end of one phase of development and the beginning of a subsequent phase directly depended on the production and approval of documents. For example, to complete the software specification phase in DoD-Std-2167 required the production of a Software Specification, and to begin the subsequent Preliminary design phase required the review and approval of that Software Specification at a Software Specification Review.

Criticisms

[Fowler, M. 2002] summarizes the complaints as follows: “These methodologies have been around for a long time. They've not been noticeable for being terribly successful. They are even less noted for being popular. The most frequent criticism of these methodologies is that they are bureaucratic. There's so much stuff to do to follow the methodology that the whole pace of development slows down.”

Efficacy

“Plan-Driven” methods have been the center of software development philosophy since organized approaches were first used in the 1970's. However, their record of success has been less than resounding. [Suzuki, 2002] summarizes the results of an ongoing study of over 8,000 software projects and 365 respondents by the Standish Group, called the “Chaos Study,” where success was defined as on-time and on-budget:

•            53% of projects were both late and over budget

•            31% were outright failures and cancelled

•            cost overruns averaged 189% of the original cost estimates

•            schedule overruns averaged 222% of the original time estimates

•            61% of originally specified features and functions were available in the released products

Documentation

[Fowler, M. 2002] observes that, largely because of the CASE tools used, software documentation often assumes a “dictionary mentality” where every aspect of the development is treated with equal importance in an effort to ensure that everything has been included. He suggests that documentation too often describes what everything “is” rather than describing what it “is meant to be” and often miring the reader in a sea of details where the essence of the development is hard to extract, particularly in complex systems.

Handling Changes

As characterized by [Noyes, 2000], change is a feature of every software development effort, and “..the [traditional method] approach was to prepare for change by trying to anticipate it all and predict what might change ahead of time. This involved performing extensive requirements analyses, capturing them all in comprehensive documentation, and reviewing them with customers and developers until everyone knew exactly what needed to be built before the work ever started.” However, [Mrenak, 1991] observes that the assumption that customers know what they want, can communicate to developers what they want, and that what they want won’t evolve over time is a fundamental error in the traditional method.

Agile Methods

The Agile Methods Manifesto

This “Manifesto” is endorsed by a number of individuals, most recognized as long time participants in the evolution of software development methods.

[Russell, 2002] presents an illuminating if somewhat biased contrast of light (Agile) methods against heavy (Plan-Driven) methods with the following recipes for “Designing a custom coffee maker:”

From this example, the flavor of Agile methods is clear: prioritize requirements, implement a subset of top requirements quickly, deliver the implementation to the customer, do it again until the customer is satisfied with the resulting software product. In general we can recognize the following characteristics of Agile Methods: 

•            the importance of individuals

•            customer commitment

•            volatile requirements

•            current-version focus

•            continuous refactoring

•            small project size

Current Variations of Agile Methods

Extreme Programming

Extreme Programming (XP) attempts to address the basic risks faced in software development including schedule slips, project cancellations, cost overruns, defect rates, mission misunderstandings, mission change, and staff turnover. There are several practices that XP utilizes: 

•            Planning,

•            Small Releases,

•            Scenarios,

•            Simple design,

•            Testing,

•            Refactoring,

•            Pair Programming,

•            Collective ownership,

•            Continuous integration,

•            40-hour week,

•            On-site customer,

•            Coding standards,

•            Open workspace and

•            Team rules

XP stresses customer satisfaction through iterative communication between the customer and the programmers. In an XP project, programmers and business managers setup scenarios to represent system requirements. Each scenario describes what will be developed and the amount of time it will take. According to a recent a study by Sanjay Murthi, using XP project teams were more enthusiastic and eager. Because of XP’s concept of pair programming, team members where able to fix anyone else’s bugs. XP also promotes regular builds, version control and working systems, which allows for early problem detection. By maintaining good version control, roll back to a previous build was an easy process if a build went bad. Software and system design in XP is kept simple and clean. Testing is performed on a daily basis to ensure a working “bug-free” system at the start of programming every day.

Rules and practices for extreme programming:

Crystal Methods

Crystal Methods is a set of human-powered software development methodologies that are collected together into a self-adapting family. The methods are based on the premise that every project needs a slightly different set of policies and conventions. The development of a project using Crystal Methods is sensitive to people issues, and improves as the people issues improve.

Crystal collects together self-adapting family of “shrink-to-fit,” human-powered software development methodologies based on the following understandings:

•           Every project needs a slightly different set of policies and conventions, or methodology.

•            The workings of the project are sensitive to people issues and improve as the people issues improve, individuals get better and their teamwork gets better.

•            Better communications and frequent deliveries reduce the need for intermediate work products.

Crystal Methodologies is a people-centric methodology that focuses on achieving project success through enhancing the work of the people involved. Crystal family methodologies work to reduce the paperwork, overhead and bureaucracy to the least that is practical for the parameters of that project. The “shrink-to-fit” attribute of Crystal refers to the fact that development starts with something small enough and continues working to make it smaller and better fitting for the project. The principle of Crystal is that

“Software development is a cooperative game – using props and markers to remind and incite, to reach the next move. The endpoint of the game is a delivered system. The next game is to alter or replace the system.” [Cockburn, A., 1999]

The description of Crystal Methodologies refers to intermediate work products. These intermediate work products are necessary to help the team make their next move in development. Their primary purpose is to help the team create the final work product [Cockburn, A., 2002]. According to Cockburn, the intermediate work products benefit the team by reminding them of the project’s purpose and to incite new ideas or thoughts that would benefit the project. Finally, Cockburn summarizes Crystal with the following:

Values: Communications using Crystal Methodologies is strong and deliverables are not extensive. Crystal “Clear” strives to keep the communications paths short, rich and informal. Have frequent Release builds. Reduce overhead. The intent is that more deliveries and better communication will reduce the need for intermediate work products.

People: For each of the roles in the software development project using Crystal Methodologies, there should be a distinct person in that role.

•           Sponsor

•         Senior Designer

•            User

•            Designer-programmers (designers)

Merged roles needed that may come from the above roles:

•            Business Expert (maybe sponsor, user or senior designer)

•            Coordinator (maybe senior designer)

•            Tester (maybe from designers)

•            Writer (maybe from designers)

Deliverables for each role:

Techniques used: Projects using Crystal Methods should be managed using milestones and risk lists. Other techniques employed during the development life cycle (i.e., project planning, requirements gathering, design, programming, testing) are decided by the team.

Dynamic Systems Development Methodology

RAD group met in 1994 to discuss definition of standard iterative process to support RAD development. [[IEEE Computer, June 2003]

The need for a Dynamic System Development Method (DSDM) came from the growing demand for Rapid Application Development technologies. DSDM is an Agile Method that is a project delivery framework that aids the development and delivery of business solutions to tight time scales and fixed budgets. In all software development projects, requirements creep has been a major barrier to conquer. To overcome this barrier, DSDM allows for requirements to change but maintains a fixed time schedule and fixed resources as much as possible. DSDM operates on the assumption that “nothing is built perfectly first time, but that a usable and useful 80% of the proposed system can be produced in 20% of the time it would take to produce the total system.” [DSDM Consortium]

The mechanism used for handling the flexibility of requirements is the time-box. The DSDM time-box is developed based on the fixed completion date of the project. An overall time frame is constructed by nesting shorter time-boxes of two to six weeks in duration. As depicted in the following figure there are three phases in time-boxing 1) Investigation, 2) Refinement and 3) Consolidation. Investigation in time-boxing is a quick pass to see if the team is taking the right direction. Secondly, Refinement builds on the comments resulting from the review at the end of the investigation. The final phase, Consolidation, ties up any loose ends in the project. Each time-box has a fixed date and a prioritized set of requirements. Requirements included in each time-box should be mixed between mandatory and of lesser priority to allow room for flexibility.

DSDM gains user acceptance early because it is an iterative process based on prototyping and involves the users throughout the project life cycle. The benefits of using DSDM include:

•            Users are more likely to claim ownership of the system.

•            Risk of building the wrong system is greatly reduced.

•            The final system is more likely to meet the users’ real business requirements.

•            Because users are involved from the beginning, they will be better trained.

•            The implementation is more likely to go smoothly because of the cooperation of all parties with vested interest in the development.

It’s important to note that users must be licensed to use the DSDM Framework. That is individuals or organizations using the Framework must be full members of the DSDM Consortium.

The DSDM Framework should be used on projects that can identify classes of users who will use the end result so that knowledgeable representatives can participate throughout the life of the project. These representatives are known to DSDM as “Ambassador Users” because they provide the two-way communication channel between the business community and the project managers. Large projects should be able to be decomposed into smaller components for either incremental delivery or for development by parallel teams. Projects employing the DSDM Framework should focus controls on ensuring that iterative development will lead safely to the right business solution.

The DSDM life-cycle consists of five phases as well as a Pre- and Post- Project phase. As depicted in the figure below, the phases are Feasibility Study, Business Study, Functional Model Iteration, Design and Build Iteration and Implementation.

The Pre-project phase, Feasibility and Business Studies are done sequentially because they set the ground rules for the rest of iterative and incremental development on the project. When the project solution is delivered, the project team is disbanded and the Post-project phase begins. This activity includes maintenance of the delivered product and checking that the expected business benefits have been achieved. Important to note, testing is not a separate function of the lifecycle because it is happening throughout both the Functional Model and Design and Build Iterations. The following table summarizes the activities in each phase of the lifecycle:

Finally, there are nine principles that are the foundations on which DSDM is based:

•            Active user involvement.

•            Team must be empowered to make decisions.

•            Frequent delivery of products.

•            Fitness for business purpose is essential for acceptance of deliverables.

•            Iterative and incremental development is necessary to converge on an accurate business solution.

•            All changes during development are reversible.

•            Requirements are baselined at a high level.

•            Testing is integrated throughout the life-cycle.

•            Collaboration and cooperation between all stakeholders is essential.

Feature Driven Development

In 1997 Peter Coad and Jeff DeLuca resurrected and restructured a Singapore logistics system project as an IID project. [IEEE Computer, June 2003]

Developed by Nebulon’s Jeff De Luca, Feature-Driven Development (FDD), incorporates several ideas and practices from the last 30 years of the IT industry. De Luca combined these varied ideas and concepts to create patterns of play that bring success [Nebulon]. FDD is a model-driven development approach that uses modeling exercises at the start of a project as a knowledge transfer and requirements, problem solving techniques, and reporting guidelines providing every stakeholder of a project with the information necessary to make sound, timely decisions. De Luca suggests that “one of the keys to good object modeling is to model the domain…” [Nebulon]. Because domain modeling involves the users, one of the outcomes of the modeling exercise is that the same language is used.

There are several processes that FDD uses. FDD starts with creating the domain object model in collaboration with the domain experts. The developers then use information gathered from this modeling activity, and other requirements activities to create a features list. After the requirements and features are known, a rough plan is developed and responsibilities assigned to various teams. These teams develop the features by repeatedly performing design and build iterations that last no longer than two weeks at a time. Each process has entry and exit criteria, tasks, and verification procedures. The original FDD processes consisted of:

•            Object Modeling

•            Features List

•            Development Planning

•            Design by Feature

•            Development by Feature

The figure depicts the FDD process flow.

The following table summarizes the latest FDD processes published by Jeff De Luca of Nebulon.:

Source: http://www.nebulon.com/articles/fdd/latestprocesses.html

Lean Development

Taken from the Japanese manufacturing concept of “Just-In-Time” development, a new development toolkit emerged, known as “lean development”. Lean development began to spread throughout many disciplines, from logistics, to the military, to construction, and to the service industry. This methodology became universally successful at improving results when applied as the building block to the agile software development practices. Not only is Lean development a way of thinking, it is a system of management that creates new operational value streams. These value streams yield physical value for the customer, which equates to revenue for the company. It is lean development that creates “knowledge value” for the company. Successful Lean Development requires an organization to make a cultural and financial investment in a Lean toolkit. It is an organizational change method that is implemented with the objective of increasing profit. One company that is highly successful at implementing lean practices is Toyota.

For Toyota, lean production was a method of organizing production using half the effort, space, inventory, and product development time when compared to mass production. In doing so, there are less defects but larger product variety. Toyota realized that the only way to increase profit is to reduce costs. The lean efforts for Toyota were aimed at eliminating all production steps for a good or service that do not add value to the final customer.

The three goals Lean Development strives to achieve is:

•            Development completion in one-third the time.

•            Complete development within one-third the budget.

•            Compete development with one-third the defect rate.

Scrum

Scrum is an agile lightweight process that can be used to manage and control software and product development. Initially formulated and presented to the Object Management Group in 1995, Scrum is a wrapper for existing engineering practices using a team-based approach to iteratively, incrementally develop systems and products when requirements are rapidly changing. The Scrum process controls the chaos of conflicting interests and needs. It improves communications, boosts morale and maximizes cooperation and productivity among for the project because of its team-based approach. Other characteristics of Scrum include: scalability, defect detection oriented, and pattern-based methodology.

Several characteristics and rules inherent in Scrum are [ADM, 1996]:

•            "It breaks down large products into manageable chunks - a few product features that small teams can create in a few months.

•            It enables project to proceed systematically even when team members cannot determine a complete and stable product design at the project's beginning.

•            It allows large teams to work like small teams by dividing work into pieces, proceeding in parallel but synchronizing continuously, stabilizing in increments, and continuously finding and fixing problems.

•            Always have a product you can theoretically ship.

•            Speak a common language on a single development site.

•            Continuously test the product as you build it.

The development process for Scrum is said to be empirical in nature because the process is only beginning to be understood, it is complex, subject to change and unpredictable. This form of empirical development using Scrum is recommended as enabling processes for developing object-oriented software or for software that has “clean interfaces”. The key to the success of Scrum is using measurements to maximize flexibility and risk while maintaining control. Control is maintained by establishing, maintaining, and monitoring key control parameters. Use of these controls is the backbone of the Scrum development process. Some controls used in Scrum are:

              •            Backlog – identification of all requirements to be fulfilled

              •            Objects/Components – self-contained reusable things

              •            Packets – group of objects within which a backlog item will be implemented

              •            Problems – what must be solved by a team member to implement a backlog item

              •            Issues – concerns that must be resolved prior to a backlog item being assigned to a packet

              •            Solutions – the resolution of an issue or problem

              •            Changes – the activities that are performed to resolve a problem

              •            Risks – the risk associated with a problem, issue, or backlog item

Scrum development uses monthly “Sprints” where each Sprint is devoted to developing features collected in a Backlog. A sprint is a set of development activities conducted over a pre-defined period, the result of which is a demonstrable executable. Duration period for a sprint is from one to six weeks. Scrum meetings (essentially triage), in which the team gathers to check the project’s progress, are held everyday. The teams in Scrum development may consist of one to seven members, among which should include a developer, quality assurance person, and a documentation member. Scrum significantly increases productivity while facilitating adaptive, empirical systems development.

The Scrum development life-cycle consists of three processes:

•            Planning and System Architecture – Product functionality and estimated delivery requirements are planned based on current backlog and assessed risk. For a new system, conceptualization and analysis are performed. However, for existing systems, the analysis is more limited. During this phase of the process, controls are established, teams assigned, and the backlog items are prioritized.

•            Closure - Based on the competition among the teams, requirements met, cost and quality, management determines if the product is ready for delivery. If the product is ready, the Sprint phase is ended and the release is declared “closed.”

•            Consolidation – The products and artifacts of the current development cycle are cleaned up for a clean start on the next development cycle. This includes opportunities for addressing those items that were omitted from development due to pressure for releasing the product.

Applications

Currently, agile methods are most commonly employed for small, web-based or client-server applications for the e-business sector. In this section, several case studies are discussed, with an emphasis on presenting experience with agile methods across a variety of different problem domains.

Financial

[Bossi, et al., 2001] describe a fairly typical application for agile methods. A financial software product was developed in which traders interact with the system via a browser to receive reports regarding risk factor and the margin position of various parties.

Several agile practices were used. The seven team members on the project were organized in an open space work area, with the computers all located on a large table-top surface in the center. Extensive use of CASE tools included: a class browser, a refactoring tool, an automated testing tool, an integration tool, and a metric reporting and calculation tool. The customer (not on-site, but available on request) was directly involved in the tradeoff of cost for function. Code refactoring consumed 50% of total development time but resulted in greater system simplicity and better self-documentation, in addition to allowing changes or new features to be implemented easily and quickly. The value of refactoring was confirmed by metrics. Another agile practice that’s value was proven was pair programming, which nearly eliminated defects. By increasing shared knowledge within the team, pair programming helped to compensate for the team’s lack of domain expertise.

The team’s overall goal was to deliver good, fast and cheap software. All required features were delivered on time, with very few defects (1). No overtime was worked during development of this product. In addition, the introduction of agile methods improved estimating and control capabilities. Furthermore, a cost reduction due to low defects was observed. The team members noted that more experience with Extreme Programming will help improve their performance as measured against their goals.

Large Project

Typically, agile methods are thought of as being primarily employed for small projects, but [Elssamadisy, A., 2001] reports on the use of agile methods on a large project by ThoughtWorks, Inc. Agile methods were initially adopted for the development of ATLAS, a leasing application, primarily due to the need to deliver a working subset of the product periodically to maintain client confidence. The project took 1.5 years and involved 35 developers, 15 business analysts and 10 software quality assurance specialists.

Agile methods in general, and Extreme Programming in particular, were not originally meant for large projects, so they were modified by project team to suit their needs. Heavy customer interaction is a salient feature of agile methods, and in fact was the reason for their adoption by the ATLAS team. However, due to the large size of the team, the business analysts interacted with the real customers and served as surrogate customers for the developers, who also depend on the domain knowledge of the analysts. Another concession to the large size of the team was the abandonment, after a few attempts, of the full day 50 person planning meeting – it was found more effective to have most of the planning done by smaller groups. An attempt was made to switch from coded functional tests to unit testing was not entirely successful, as a large project requires a large test suite to ensure complete coverage. Ultimately, a compromise decision was made to go to automated functional tests. Refactoring is a definite need on a large project to clean up inconsistent code. It was painful at first, because of the large code base developed before the adoption of agile methods, but as the project went on, the code became more clear and consistent. Refactoring takes a long time and tends to get pushed off near deadlines. Time needs to be allotted to refactor the design as well as code. Pair programming was used for new functionality, but not used for more straightforward tasks such as debugging, maintenance, and refactoring. Some people don’t like pair programming, for example, some of the most talented developers felt that it slows them down, while others just don’t work well with others. Major compromises had to be made to the collective ownership / information sharing aspects of agile methods to accommodate the large team size. Of course, these characteristics are necessary to certain extent to keep the project from fragmenting, but it was impossible for each developer to completely grasp the entire product. Small groups of developers specialized in certain areas. On a large, complex project “getting up to speed” takes time. Some internal coding standards emerged over time, which helped, and regular presentations to keep the team up to date on the big picture were beneficial as well. At the beginning of the project the 40 hour week was more of a goal than a reality, with 50-60 hour weeks the norm just before each release was due. With more experience, the developers became better at estimating time needed for each iteration, and the 40 hour week became normal. The need for frequent releases was the main driver for the adoption of agile methods for this organization. Generally two week iterations were the norm, as they were found to be better for estimating purposes and the right size for new functionality to be added. However, some tasks required longer so the team needed to be flexible, and allow longer duration iterations when necessary.

Clinical

An agent is an encapsulated computer system capable of flexible, autonomous action to support its principal. [Knublauch, et al., 2002] describe the use of Extreme Programming in the development of a clinical multi-agent system that will proactively deliver critical patient to the responsible physician at the appropriate time. Software agents, particularly in a multi-agent system, are characterized by their flexibility, their autonomy and individual viewpoints, and their small size and loose coupling.

A variety of agile methods were implemented in this case. Heavy customer involvement was necessary, since the requirements were weakly specified, so the clinical staff was closely integrated into the development process. The customer, a medical doctor, was onsite for the duration of the project. An unusual metric was recorded – the onsite customer averaged one question per hour. All the developers were seated around a conference table in a single room. CASE tools that were used included a code generation/refactoring tool, an automated unit testing tool, and an environment for configuration management, continuous integration and simulation. The 40 hour week was strictly adhered to , as it was observed that pair programming is exhausting. Each pair worked on an agent, focusing on those agents identified by clinical expert as having greatest value. Pairs rotated members and agents frequently. This arrangement turned out to be highly productive and contributed to spread of clinical knowledge among the team members. The relaxed atmosphere stimulated open, honest communication, which was particularly important in this application, where there was a need to prevent communication barriers between developers and clinical experts. Testing consisted of a set of accumulated unit tests throughout the course of development to clarify misunderstanding between the developers and the clinical experts. The implementation of coding standards was enabled by the use of a code generation tool for automated code layout. The team’s deliberate focus on programming rather than design was suitable for agents given their small size and few tasks. The small size of the agents also made refactoring easy as well as making it possible to rewrite them from scratch without much penalty in terms of cost or time. Continuous integration was managed by uploading the agents to a server with simulation environment as they passed the unit tests, where the agents were tested every evening.

Extreme Programming is a natural fit to multi-agent systems consisting of small autonomous units that can be implemented, tested, and refactored rapidly. Complex interaction between the agents as well as their emergent behavior makes preplanning difficult. In addition, the close involvement of domain experts, necessary in the clinical environment is an inherent part of this agile method.

Maintenance

It is commonly accepted that the maintenance of existing software constitutes the majority of the total life cycle cost. [Poole, et al., 2001] describe the use of Extreme Programming for this application. Although agile methods are usually associated with new software development, it will be shown that their adoption for the maintenance of existing software can provide substantial benefits.

Iona Technologies’ Orbix is a Corba based middleware product in the maintenance phase of its lifecycle. A forty person team performs bug-fixing and enhancements to the product. Some Extreme Programming practices are a natural fit to maintenance team, for example, incremental development. Each bug-fix or enhancement corresponded to one user story (about a two week iteration). Larger tasks were either broken up into several smaller releases or, in some cases, a longer time increment up to four weeks was allowed. Another Extreme Programming practice that proved to be a natural complement to maintenance was the “write a test first, watch it fail, then code” procedure. Essentially, the bug report served as the first iteration of this cycle. Testing is critical in maintenance, as it is in Extreme Programming. An automated test suite was adopted, and for each bug or enhancement one or more tests were added to the test suite.

Other Extreme Programming practices that were utilized included a daily stand-up meeting lasting 15-30 minutes. These meetings were found effective because the engineers reexamine their use of time when they have to explain why they are taking so long on a particular task or why they are working on a low priority task when higher priority task remains undone. The team also the value of refactoring - if it isn’t done each “band-aid” increases code entropy and increases the likelihood of introducing regression faults. The work area consisted of a large tabletop where the team’s computers were all located, surrounded by whiteboards and flipcharts, resulted in an increase in the visibility of what team members were doing, as well as an increase in communication between team members.

Some modifications to doctrinaire Extreme Programming were made in order to accommodate the unique requirements of the maintenance environment. Pair programming was not formally introduced, both because of the less conceptually complex nature of the work, as well as the difficulty in convincing engineers of its practicality. Some pair programming did occur when senior staff mentored junior engineers, where it was found to be useful. An onsite customer was not possible, since most of the bug fixes and enhancements affected multiple customers. Instead, product managers and customer service representatives acted as customers.

There were some aspects of Extreme Programming that couldn’t be implemented. Collective ownership of a large product is difficult, even more so given the fact that maintenance is usually assigned to less experienced software engineers, so team members had to specialize. The principle of simple design is outside the scope of maintainers, who generally try to preserve the existing design. No explanation was given as to why the forty-hour work week was rejected by management - the reader is invited to draw his or her own conclusions as to the state of labor relations in the software industry.

Iona Technologies measured a 67% increase in the number of bugs closed per week after the introduction of its variant of Extreme Programming. On a more qualitative note, increased customer satisfaction with the maintenance team’s greater responsiveness was also observed.

Embedded

Embedded systems are not the first application that comes to mind when discussing agile methods, but [Grenning, J., [2001] describes an adaptation of Extreme Programming for an embedded system application at a process-intensive organization.

Most of the developers at this organization had one to three years of experience. In order to compensate for this lack of experience the organization employed a traditional waterfall life cycle model including substantial up front work: requirements documentation, design documentation, and a formal review process in which a few senior software engineers would review and approve the work of the junior software engineers. All this up front work resulted in the generation of substantial overhead costs and unrealistic deadlines. In addition, it was difficult to cope with changes or surprises late in a project, often resulting in bugs in the final product, late delivery, and engineers getting burned out from long hours before a deadline.

Management was interested in the benefits of agile methods, but the organization’s reliance on up front work, reviews, and documentation meant that Extreme Programming was not acceptable. A consultant developed a compromise approach, focusing on the goals that could be achieved: faster and more predictable development of a reliable high quality working product, more job satisfaction for developers, and enough documentation for efficient maintenance.

A “little design up front” approach was adopted to meet the organization’s needs. Monthly design reviews and hand-drawn diagrams complemented a reduced set of high-level documentation, which would not need to be changed after each maintenance activity. The documentation was the minimum sufficient to point the maintainer to the right section of source code, which would now be more understandable.

Some Extreme Programming practices were adopted outright. A systems engineer acted as the onsite customer. Small releases (on the order of one month), automated unit tests and continuous integration helped keep bugs from creeping into the product. Collective ownership, refactoring and company coding standards ensured that the product would be maintainable. Pair programming allowed senior developers to spread their knowledge to the less experienced team members. The forty-hour week, made possible by higher productivity, kept up morale and prevented burn out. Overall, the introduction of practices associated with agile methods met its goal of enabling the organization to adapt to changing requirements.

Safety-Critical

[Canos, et al., 2002] discuss an unusual application of agile methods – a complex hypermedia application designed to improve safety in public transportation. The procedures for handling emergencies in the underground metropolitan transportation system for Valencia, Spain were converted from a text document in order show on a single screen all the information needed to make a decision, with an emphasis on reducing response time.

Formal plan based methods were not considered suitable for a number of reasons. This high risk critical application required extensive testing (considered absolutely crucial) during development. As a result of the testing, continuous improvement would be necessary. As the project consisted primarily of computerization of a text-based emergency plan, no redesign of the basic structure was needed, nor was specific domain knowledge necessary. The large number of special cases became apparent after the initial design, meaning that changes on the fly and a short feedback loop would have to be handled.

Many of the practices of agile methods were put into use by the project team. To ensure the consistency of the information in the emergency plan, incremental development and heavy customer involvement were a must. The project started with a simple high-level conceptual design consisting of an abstract representation of the user interface and an object-oriented model. Pair programming teams normally were made up of one developer and one domain/safety expert, although a trio of two developers and the domain/safety expert was sometimes employed. Collective ownership was not always possible with such a large project, so some division into smaller groups with localized responsibilities was necessary. This application of agile methods shows how much they have to offer even seemingly incompatible problem domains.

Criticisms

The literature enumerates a variety of criticisms of agile methods. Some of these criticisms focus on the perception of agile methods as “hacking”. Others focus on the lack of quantified, documented results, reliance on the most talented developers, and over-reaction to change.

[Johansen, et al., 2001] describe a number of reasons for the failure of agile methods to gain broad-based acceptance in the software industry. These reasons revolve around the advocates of agile methods’ inability to relate to the needs of management and the difficulty of measuring the impact of a new methodology. The problem starts with the name Extreme Programming. “Extreme” is not a word that management likes to hear. It suggests that this method is not to be associated with a serious software effort. Although disagreement over a name might seem like a minor point, it illustrates how the proponents of agile methods are somewhat out of touch with the needs and outlook of management. The forty-hour work week seems lazy and indicative of a lack of commitment on the part of the developers. Similarly, pair programming is perceived as less productive. Pair programming is not attractive to many developers, including the very talented, who are slowed down by this arrangement, and those who don’t work well with others. In practical terms, the emphasis on testing requires a CASE tool for automated unit testing, which can present an additional barrier for organizations that do not currently use such a tool. Most agile methods reject formal review processes such as inspections - a potentially valuable method of detecting faults in a product.

Proponents of agile methods have also had difficulty measuring its impact. Agile methods largely focus improvement on hard to quantify or meter “people factors”, so much of the evidence in their favor is anecdotal, and therefore subject to differences in interpretation of the same situation by different people. Also, there are too many variables, which also leaves room for this subjectivity. For example, the embedded system Extreme Programming project, despite only positive qualitative results reported by the source, was canceled, according to the source, for “political” reasons.

[Alleman, G., 2002] criticizes agile methods on the grounds that they are too general, and therefore difficult to apply. He also criticizes over-agility, arguing that unstable business requirements indicate a larger problem within the organization, and being able to deal with the symptom will not solve the root problem. [Boehm, B., 2002] agrees on this point, attributing a $3 billion overrun on the FAA’s Advanced Automation System for air traffic control to the project team’s over-reaction to change. Alleman points out that rejection of documentation other than working code can create missed opportunities to communicate high-level information or resolve problems at an early stage. He also highlights a weakness of agile methods in their reliance on highly skilled and motivated developers, without addressing the shortage of such individuals in many organizations. Boehm agrees, citing an unidentified 1975 survey of fourteen large industry and software organizations. The survey provides a profile of an “average programmer” having two years of college education, two years software experience, familiarity with two programming languages, and qualitatively sloppy, inflexible, introverted, working beyond his or her level of competence, and under managed. No data is reported since 1975, but given everything that has happened in the software industry since then the demand vs. supply of experienced, highly skilled developers has certainly not improved.

The Future of Agile Methods

A Middle Ground

A respected voice of moderation has weighed in on the effectiveness of Agile methods. Barry Boehm discusses the consequences of using both Agile methods and Plan-Driven methods over several aspects of software development in [Boehm, 2002].

According to Boehm, both Agile and Plan-Driven methods have responsible centers, that is, advocates and practitioners that recognize the limitations and advantages of both and try to apply them appropriately. In spite of criticisms to the contrary, planning, Boehm observes, is not ignored in Agile methods: “Compared to unplanned and undisciplined hacking, agile methods emphasize a fair amount of planning.” The distinction with Plan-Driven methods is that Agile methods do not strive to produce the documentation that Plan-Driven methods emphasize. Instead, Agile methods depend on the memories and talent of the team members to capture that planning and apply it appropriately. Boehm compares the two methods from several perspectives:

Developers

The quality of the developers is important to both methods. However, Boehm observes that, in lieu of written plans, agile methods rely on the knowledge of “premium” quality developers and their “tacit” ability to understand the trajectory of rapidly changing requirements to a degree sufficient to avoid wrong architectural decisions. If these “premium” developers are up to the task, then prospects are good. But if they’re not, and in particular if they don’t know they’re not, then disasters are more likely.

On the other hand, traditional methods, with documented plans, can expect timely corrections to architectural mistakes because outside experts can review them. However, the time required to implement this “expert” feedback mechanism can easily render these documented plans obsolete in a reality of rapidly changing requirements.

Customers

Agile methods emphasize continuous customer involvement, while traditional methods expect periodic customer involvement at formal milestones. For both methods, Boehm suggests that unless customers are “..committed, knowledgeable, collaborative, representative, and empowered ..” software products are unlikely to be used even when they meet “requirements.” Boehm cautions that agile methods may be more vulnerable to inadequate customer participation because there are no external means (that traditional methods facilitate with written plans) for detecting problems.

Requirements

Just how fast are requirements likely to change? Advocates of agile methods suggest they work better than traditional methods when requirements are volatile, much greater than 1% per month. Boehm agrees and sees great potential for agile methods to succeed in such environments where traditional methods would surely bog down handling formal changes.

But there are risks and paradoxes. What if agile developers mishandle the adaptation of changes? Without formal traceability from requirements to code and back, the potential for change confusion and rework is high. And what if the system is safety or money critical? Who would trust their lives or life savings to software developed without formal and detailed oversight? Yet how can formal and detailed oversight be applied to a problem whose features and risks are rapidly evolving?

Architecture

To construct an architecture for a software product is, in traditional theory, to provide a stable foundation for the evolution of the product, both during its initial development and also during its subsequent operational phase. Good architectures facilitate the addition of future functionality because they have, through planning, anticipated that functionality to a useful degree. In contrast, Boehm observes that agile methods value “..maximizing the work not done..” and notes the Extreme Programming practice of actually expending effort to eliminate architectural considerations that don’t support the current version (current functional requirements) of the product.

When requirements are volatile, you can reasonably expect that minimizing or ignoring architectural considerations will do little harm since no one can know with any predictability which architecture is best suited for the future. However, when requirements are not volatile, you can also reasonably expect that minimizing architectural considerations is almost certainly going to result in rework.

Refactoring

Refactoring is revising and simplifying the design or implementation of a software product without disturbing its functionality. It is, essentially, a series of mini-cycles within the larger phase that refine the initial design or code solutions into better ones. For agile methods “better ones” are those without architectural considerations, so refactoring is really built into the process.

But what does refactoring contribute to the quality of the product? Agile methods are built on the premise that the contribution is positive. Boehm concludes from his analysis of very large software development projects, however, that most rework costs (80%) were the result of poor architectures, where no amount of refactoring would have helped.

Size

Can agile methods be applied to larger projects? Most advocates currently prescribe them for smaller projects (less than 25 people), although recently there have been some suggestions that agile methods can be adapted to larger (50-250 people) projects [Fraunhofer 2002]. Most however have serious reservations about using agile methods for projects that involve many hundreds or thousands of people ... and the generally agreed reason is communication. Agile methods depend on intimate and frequent communication between all members of the project. In Extreme Programming, for example, the principle of group code ownership assumes that everyone in the project can and will acquire a meaningful understanding of everybody’s code. But when many hundreds or thousands of people are involved, intimate and frequent communication is not attainable.

Balancing the Risks

Boehm believes there are “home grounds” for each method; circumstances and conditions where one, and not the other, should be applied. “Particularly in the e-services sector,” he observes, “companies with a large customer base don’t need just rapid value or high assurance -- they need both.” Table 2 paraphrases Boehm’s characteristics and parameters for the “home-ground” of each method.

But how much of each? Or, more to the point, for a particular circumstance, how much planning is needed to balance the dual demands of early product delivery and sufficient product quality?

Boehm illustrates these competing demands in terms of risk exposure as a function of the time and effort invested in plans. If you spend little time and effort to plan, then the risk exposure for the goal of high product quality increases, while the risk exposure for the goal of early product delivery decreases. On the other hand, if you spend a lot of time and effort to plan, then the risk exposures reverse: it becomes more doubtful that you will get the product delivered early, but more certain that its quality will be high. Between the extremes of low and high investments in planning, Boehm suggests a “sweet spot” where you can find a minimum value for total risk exposure in projects where a combination of agile and plan-driven methods are appropriate.

 

The Unanswered Questions

In the long run, the success or failure of agile methods will depend on whether and where they produce a better ROI. Can Agile methods help correct the long-known deficiencies of Plan-Driven methods? For each unit of cost and effort expended, given particular developers, customers, requirements, domain, and size, do agile methods produce better software products? The critical questions are easy to frame:

•            do agile methods produce software that has fewer defects?

•            do agile methods produce software that is more easily maintained?

•            do agile methods produce software that is more usable?

•            do agile methods produce software that has better integrity?

There has been comparatively little experience with agile methods to date. In fact, the practices and principles of agile methods have not yet been entirely formulated [Fraunhofer 2002] and that will have to occur before meaningful tests and measurements to answer these questions can be undertaken.

Will Agile Methods Work For Average Developers?

[Boehm, B., 2002 (2)] notes that agile methods depend on highly skilled developers but that is not always an accurate description of the project team. [Rajilich, V., 2001] suggests an agile method for novice programmers based on incremental development, which he feels is easier for the less experienced programmer to comprehend. Each increment adds functionality closely related to some domain concept, thus determining the order of incremental enhancements. Incremental development, the fast delivery of small releases, reduces risk because problems can be caught early on.

How Good Are Agile Methods?

[Reifer, D.J., 2002] conducted a survey of fourteen firms using agile methods. The firms ranged across ten industry segments, but were mostly advanced organizations (CMM level 2 or greater) with a poor record of delivering high quality software products on-time and within budget. These firms were willing to try something new to solve these problems. E-commerce and e-business firms were heavily represented. Most of these organizations were conducting pilot projects to study agile methods and planned on integrating agile practices that worked well into their regular software development processes.

The pilot projects, basically experiments, were mostly small, usually with less than ten participants. These projects typically featured an in-house customer, established requirements, a stable architecture, low-risk, and a high degree of development flexibility. Given these characteristics it is not surprising that they were usually quick to market (one year or less) web-based or client-server applications. The projects were staffed with motivated, experienced (but relatively young) high performers.

There was some commonality of the practices implemented in the various projects. A cyclical process was characterized by collaborative development with full customer participation. The emphasis was placed on a working product rather than reviews or throwaway prototypes. Disagreement existed in terms of the actual process used , with spiral, incremental, and other types all seeing some application. There was also some variation regarding the level of informality or flexibility, as well as who stakeholders were and what their level of involvement was.

Five of the organizations kept metrics and benchmarks to enable a quantitative evaluation of agile versus plan driven methods. Some of the results were:

•            15 – 23 % improvement in productivity

•            5 – 7 % reduction in cost

•            25 – 50 % time to market compression

•            defect rates on par with other projects            

All of the participants in the survey argued for the use of agile methods based on qualitative factors such as recruitment and morale. The biggest problems reported in the survey were general issues regarding technology transfer from the pilot project to the organization as a whole, given the particular characteristics of the pilot project team and how they differed from those of the entire organization.

References

Alleman, G., 2002. “Putting Agility in Context.” Information Age, October 8, 2002.

Ambler, S.W., 2002. "Agile Modeling and eXtreme Programming." http://www.agilemodeling.com/essays/agileModelingXP.htm

BMP 2002. Http://www.bmpcoe.org/guideling/books/2167

Boehm, B., 2002. “Get Ready for Agile Methods, with Care.” IEEE, Computer, January 2002, pp 64-69.

Boehm, B., 2002. “Agile and Plan-Driven Methods – Oil and Water?” USC, presented at Agile Universe 2002, August 2002.

Bossi, P. Cirillo, F. 2001. “Repo Margining System.” http://agilealliance.com, May 2001.

Canos, J.H., Jaen, J., Carsi, J.A., Penades, M.D., 2002. “On the Use of XP in the Development of Safety-Oriented Hypermedia Systems.” XP 2002, May 2002.

Cockburn, A., 1991. “Software Development as a Cooperative Game.” Humans and Technology Conference, Salt Lake City, UT, 1999.

Cockburn, A., 2002. Draft: Crystal Clear: A human-powered software development methodology for small teams, November 1998.

Cockburn, A., 2002. "Learning for Agile Software Development - Part One", CrossTalk, October 2002.

“Controlled Chaos: Living on the Edge”, Advanced Development Methods, Inc., 1996.

De Luca, J., 1993-2002. “Agile Software Development using Feature Driven Development (FDD)”, http://www.nebulon.com/fdd/index.html

Demarco, T., 2002. “The Agile Method Fray.” IEEE, Computer, June 2002, pp 90-92.

Eischen, K., 2002. “Software Development: An Outsider’s View.” IEEE, Computer, May 2002, pp 36-44.

Elssamadisy, A., 2001. “XP on a Large Project.”, presented at XP Universe 2001, August 2001.

Fowler, M., 2002. Http://www.martinfowler.com/

Fraunhofer, 2002. Records of discussions from eWorkshops on Agile Methods held 18 April and 19 June 2002 at the Fraunhofer USA Center for Experimental Software Engineering at the University of Maryland, http://fc-md.umd.edu/projects/Agile/

Grenning, J., 2001. “Launching Extreme Programming at a Process-Intensive Company.” IEEE Software, November/December 2001.

Highsmith, J., 2002. “What is Agile Software Development”, CrossTalk. October 2002.

Johansen, K., Stauffer, R., Turner, D., 2001. “Learning By Doing: Why XP Doesn’t Sell.” XP Universe 2001, August 2001.

Knublauch, H., Koeth, H., Rose, T., 2002. “Agile Development of a Clinical Multi-Agent System: An Extreme Programming Case Study.” XP 2002, May 2002.

Mrenak, G.M., 1990 “Evolving Concepts or Why Users Often Don’t Recognize the Software They Asked For,” Proceedings of the Seventh Washington Ada Symposium, 25-28 June 1990.

Murthi, S., 2002. "Scaling Agile Methods, Can eXtreme programming work for large projects?" New Architect, October 2002.

Palmer, S., “Feature-Driven Development to the Rescue”, www.step-10.com.

Poole, C., Huisman, J.W., “Using Extreme Programming in a Maintenance Enviornment.”, IEEE Software, November/December 2001.

Rajilich, V., “A Methodology for Incremental Changes.” XP 2001, May 2001.

Reifer, D.J., 2002. “How Good are Agile Methods?” IEEE Software, July/August 2002.

Ntafos, S. 1997 “The Cost of Software Failures.” http://www.utdallas.edu/research/esc/publications/ntafos_iasted97.doc

Russell, L., 2002. “Agile Development: A Tale of Two Cooks” http://www.russellmartin.com/articles/agile%20development.htm

Noyes, B., 2002. “Face Change With Agile Methods. Change is inevitable in software development. Agile methods help you deal with it.” http://www.fawcette.com/resources/managingdev/methodologies/agile/, posted 19 June 2002.

Schach, S.R., 2002. Object-Oriented And Classical Software Engineering, McGraw-Hill, 2002.

Schwaber, K., 2002. "Silver Bullet or Bitter Pill?". Advanced Development Methods, Inc., October 2002.