Guidelines for Gathering Requirements It is important to keep the requirements at a consistent level of detail and to be thorough. That said, it is also important to make sure the requirements-gathering process does not become an end unto itself. Striking this balance between too little and too much will result in a solid foundation for the entire development process. The following guidelines give general advice on the requirements-gathering process as a whole. Focus on the Problem Remember to concentrate on what the system does, and ignore questions about how it works. For example, it is inappropriate to choose between two competing technologies in a requirements document, but it is necessary to include any known deployment constraints. The customer may require access to certain functionality from a Web browser, which is a requirement and so must be noted for each use case. However, the decision to use Java servlets as opposed to JavaServer Pages (JSP) is inappropriate in a requirements document. Later, when the project focuses on technology selection and architecture, the requirements provide valuable information that shapes these difficult and crucial decisions. Attempting to make the decisions while still gathering requirements leads to hasty decisions that are based on incomplete information. Don't Give Up Perseverance is often the hardest part of use case modeling. A large system may have dozens of use cases with several flows of events for each use case. There is an activity diagram for each use case. Be complete, and strive to maintain a consistent level of quality and detail. The collective understanding of every system evolves over time, and requires countless hours of review of the meeting notes and updates to the use case model. It often helps to develop the requirements in stages. First, develop the high-level use case model, which identifies the actors and use cases. Validating this model with the user builds consensus and clarifies the boundaries of the system. Next, develop descriptions and flows of events for some strategically important use cases. It may be appropriate to review this material with some of the stakeholders as you proceed. Continue this process until you have at least a description and the normal flow for each use case in the model. After validating this model with the stakeholders, identify a few use cases for further exploration and dig in. At this stage, it is appropriate to spend several days on each use case, developing alternate and exception flows of events and consolidating the flows in an activity diagram. After a few use cases have elaborated, it may be necessary to refactor the high-level use case model, using techniques such as the include and extend relationships between use cases or use case generalization. After reviewing this block of work with the stakeholders, you must decide to design and develop the well-defined use cases or elaborate additional use cases. A commitment to complete requirements does not preclude other concurrent efforts. For instance, once each use case has been described and reviewed, parallel efforts for screen design, technology selection, and architecture may coexist with refinement of the use case model. Obviously, these processes must not start too early; but judicious overlapping of these activities is relatively safe and often provides valuable insight. On the other hand, it is very dangerous to consolidate the various documents or consolidate the efforts. Each effort has separate goals and deserves a separate schedule and separate reviews. Don't Go Too Far Authors and speakers, myself included, use statistics to frighten developers into gathering comprehensive requirements. Having seen the effects of inadequate requirements, I believe that this tactic is justifiable. Nevertheless, requirements gathering can be overemphasized and drawn out past its usefulness. Remember, requirements are not an end unto themselves. They are a foundation for the rest of the development process. Foundations must be strong and durable; they do not need to be artistic masterpieces. In some cases the requirements-gathering process drags on and on due to a reluctance to move forward or a desire to achieve perfect requirements. This tendency may be exacerbated by a demanding or indecisive customer or by uncertainty about the next step. Developers are especially vulnerable on their first project with a new methodology or with new technology. I have seen some projects become mired in requirements long after the benefits stopped, because it was easier to continue to impress the stakeholders with another round of requirements than to face an uncertain future. Of course, this never works out well, as the next step must eventually be taken, and time spent endlessly refining requirements cannot be used on prototyping or technology-training that might remove some of the uncertainty. Believe in the Process Solid requirements are essential to your project's success. Without them, you are doomed to solve the wrong problems, repeat your efforts, and anger your customer. Attempting to save time by neglecting requirements invariably leads to extending the duration of the project and higher costs. As Benjamin Franklin put it, "Haste makes waste." Discovering a missing or misunderstood requirement during system test is often catastrophic. Fixing an omission or mistake may require weeks or months of development. In the meantime, marketing schedules are destroyed, expensive production hardware sits unused, business opportunities are missed, and credibility is lost forever. In some cases, projects, jobs, even entire companies are lost. At best, developers pour their hearts into a year's effort and produce a system that the customers think they can live with, although they don't really see what that one screen is for and they really wish that it worked differently. Poor requirements lead to project failure in several ways: Customer confidence in the development process and the developers erodes with each misunderstanding. Each failure raises the pressure for the next attempt; hence, communication between developers and stakeholders deteriorates rapidly. Once communication breaks down, the misunderstandings grow, and an adversarial cloud settles over the project. It is very difficult to reverse this trend once it starts. The software base bloats as it expands to accommodate an ever-changing view of the system. New requirements and new understandings of existing requirements add layer after layer to the implementation. Continued over months and years, this process can mean a tedious death of a project by a thousand cuts. Developer morale collapses as they realize that they are spending more and more time but accomplishing less and less. Each time a developer spends time solving the wrong problem, his or her faith in the process and hope for success diminishes. On the other hand, a good use case model is a solid foundation for the rest of the project: Developers begin analysis, architecture, and design with confidence. They know that the requirements will not shift radically beneath their feet, and they have a clear source of information as they proceed. Certainly they expect to ask questions and refine the requirements as they go, but the core value is complete. This allows them to focus their attention on the solution. Technology selection and architecture decisions are based on complete information. Also, since the overall problem is well understood, developers may comfortably focus on a few use cases at a time. Project managers treat each use case as a unit of work for scheduling, risk tracking, and estimation. Developers produce effort estimates and identify risks for each use case in turn. This allows the project manager and the senior technical staff to develop a very sophisticated project plan that identifies and attacks risks early in the project, by targeting particular use cases. For example, one use case may be identified as a high priority due to its complex processing or to a challenging user interface. Also, many project teams are new to the solution technologies. Selecting simple use cases for the initial development effort may help alleviate this common and serious risk. It is better to build on success than to overreach and live with failure. So, project managers should leverage the use case model as they estimate and schedule development, manage risks, and grow their team. Testers treat each use case as a basis for a section in the test plan. A well-written flow of events form easily evolves into a test case. Remember, the flow of events describes the interactions between the actor and the system. In the test case, the actor's requests and inputs become test directions, and the system's responses become the expected result. Of course, the use case is written without consideration for particular screens or technology, so additional instructions must be added. However, the basic structure and narrative flow often remains unchanged in the evolution from flow of events to test case. This creates an interesting side effect: People with experience developing test plans may be very proficient at reviewing and writing use cases. Stakeholders track progress objectively; either the latest release supports a use case or it doesn't. Objective measurement tends to reduce tensions between developers and stakeholders. Clearly understood bad news is still better than being completely in the dark. Completing three out of the five use cases is a clear, if partial, success. Everyone knows what is complete and what is not complete. Compare this to the equivalent measurement of a milestone being 60 percent complete. Stakeholders tend to feel frustrated and mislead, as the system progress is a mystery to them, and the percentcomplete numbers never quite move as expected. How to Detect Poor Requirements If gathering requirements is key to project success, then everyone must be doing it. Tragically, this is not the case. Many organizations, both large and small, skim past requirements or develop them so poorly that all benefit is lost. There are several common rationalizations for this negligence: Excessive schedule pressure No clear vision Premature architecture and design The following sections describe these common paths to poor requirements, including symptoms and remedies. Path 1: Excessive Schedule Pressure Excessive schedule pressure is the most common rationale for skipping or skimming through the requirements-gathering process. It is also the worst possible combination: The development team does not know what it is trying to accomplish, but they are working too fast to notice. The pressure to get something, anything, out the door is overwhelming. In the end, very bright people spend nights and weekends paying the price. There are two distinct variants on this path. In one, management is almost completely uninvolved. Their only concern is the final milestone, so the constant refrain is "When will you be done?" The alternative version is micromanaged pressure. In this version, management obsessively tracks the schedule and misuses schedule-oriented practices, such as commitment-based scheduling and time-boxing. Symptoms Hard deadlines are determined before the objectives are defined. Developers make feature decisions as they code. Everyone is coding, but there is no coherent or readable requirements document. Solution Education, redirection of energy, and verification are the keys to avoiding this path. Managers, developers, and the stakeholders must understand the process and importance of gathering requirements; and requirements reviews must be used to ensure that requirements are complete before moving forward. If the schedule is truly important, managers and the entire development staff must redirect their energies toward practices that do work. First, both developers and managers need to understand that trying to save time by skimping on requirements is a dangerous and counterproductive practice. Reinforce the following points: Incomplete requirements lead to expensive and time-consuming rework. Work that is invalidated by a new requirement or a new interpretation is completely wasted. The longer it takes to catch missing or incorrect requirements, the longer it will take to undo the damage. Committing to gathering solid requirements prevents rework, thereby shortening the actual schedule. Fostering positive relations with the other stakeholders during requirements gathering sets the stage for win-win compromises on schedule and features. Once everyone understands the need for good requirements, they must commit to producing them. Specifically, managers must schedule time for requirements gathering, and ensure that the requirement gatherers have access to the right stakeholders. The requirement gatherers must be thorough and persistent. Requirements must be reviewed for quality and completeness at scheduled intervals. At least some of these reviews must include a wide range of stakeholders. Gathering requirements takes skill and a great deal of persistence. With a little practice and a lot of willpower, every project team can excel at this process. Path 2: No Clear Vision In this situation, the development team gathers requirements for a poorly defined system. Without a solid system scope and vision, requirements tend to change and grow at each requirements meeting. If there is no clear system sponsor, requirements change as developers attempt to please various potential sponsors. A lack of vision often leads to requirement artifacts of unusually large size. These intricate creations grow, but rarely improve. I have seen over $1 million (U.S.) spent gathering requirements and prototyping, all without a clear vision of the system. At best, the time is simply wasted. At worst, the final set of requirements actually impedes progress because it is burdened by obsolete or contradictory requirements. Ironically, the requirements-gathering process and the developers often receive the blame. Symptoms Requirements documents are very large and very convoluted. Requirements are contradictory. Frequent changes are made to requirements. Requirements are unstable: They are included, excluded, and reinstated in a bizarre birth, death, and reincarnation cycle Corporate politics are an integral part of requirement meetings. Solution Developers can raise awareness of the risks associated with gathering requirements without a clear vision. Unfortunately, the underlying political barriers to commitment may be very real. Perhaps the budget is fragmented among different departments. Perhaps the most likely sponsor swore off commitment to risky software development projects as part of a New Year's resolution. The first goal is to find or create a sponsor. In some cases, there are stakeholders who face severe consequences if the project does not succeed. These stakeholders may listen to reason and become the sponsor. As always, finding receptive people and educating them about risks and dangers is a delicate task. But the alternative is worse. If no sponsor can be found, the developers may still be able to bring some order to the chaos. Rather than having one set of requirements that has pieces of each vision, try to maintain a distinct document for each major vision. This may help the stakeholders see the dilemma. At the very least, the developers' lives will improve, as each document stays relatively simple. Path 3: Premature Architecture and Design In times of adversity, we revert to what we know and believe. Is it surprising when developers forsake tedious and often contentious requirements meetings in favor of technology selection, design, and code? It is difficult to spend endless hours establishing the details of the system requirements when design and code beckon with the promise of quick progress, objective results, and intellectual stimulation. As a result, requirements documents either atrophy from neglect or evolve to resemble architecture and design documents. Either way, the quick progress is often illusory, because it is progress in a poorly defined direction. Symptoms The requirements-gathering process ends, with no closure or consensus. Requirements documents contain implementation details. Solution There are two solutions to this situation. First, educate and encourage developers to make requirements a priority. Second, make gathering requirements as easy as possible for developers. Developers must understand the benefits of requirements and the dangers of developing software without them. Reinforce the following points at every opportunity: Incomplete requirements lead to poor technology selection and architecture decisions. The time spent making these decisions and implementing them can- not be recovered. Solid requirements make estimation, scheduling, and testing possible. All of these activities help the developer coexist with the other stakeholders, by baselining expectations and providing project visibility. These messages are especially powerful when combined with commitment-based scheduling. In this scenario, a developer develops requirements for part of a system, then produces an effort estimate based on the requirements. This allows developers to benefit from their own diligence. In many organizations, gathering requirements is unnecessarily burdensome for developers. For instance, tedious formatting standards can be relaxed or the work may be shifted to other personnel. Dedicated business analysts can interact with the customer and start the process. The Next Step This chapter established a number of steps and guidelines for gathering requirements, including gathering raw requirements, creating a high-level view of the requirements, and describing the details of the requirements. Chapter 3 introduces a simple sample application to demonstrate how you can use these techniques.