The hardest single part of building a software system is deciding precisely what to build. No other part of the conceptual work is as difficult as establishing the detailed technical requirements, including all the interfaces to people, machines, and other software systems...
– Brooks, Fredrick P., (1986), "No Silver Bullet – Essence and Accident in Software Engineering" Proceedings of the IFIP Tenth World Computing Conference: 1069-1076
The Problem
Requirements, or in the case of an Agile project, user stories, document the capabilities you want in a planned system. And sometimes, the development of those user stories is the most difficult phase of the entire project. In order to complete this phase, the user has to have a clear picture of what he wants the system to do. She must also have an excellent understanding of their job and its daily activities.
Traditional Requirements Development
In a traditional, or waterfall software development process, the requirements are gathered at the initial stages of the process. The very first set of requirements can be developed as a part of a concept of operations (ConOps). These will usually be very high level, covering a general capability or function. And they will also be based on a limited understanding of what the system will need to be.
Later, through user groups, site visits, interviews, and other activities, more specific requirements are developed. They will usually take the form of "The System SHALL..." The requirements discuss what the system is to do, never how it is to do it. As the requirements are refined, they will be decomposed from higher level requirements to lower level requirements; from "the system shall track satellites" to all of the activities a system must do in order to accomplish the required tracking.
A design is developed for the system. Once approved, the coders develop the software and interfaces. Once completed, the code and hardware are tested. And somewhere around here, the user/client gets to see the product.
Sometimes the user is unhappy because their business needs have changed since the development process started, or the system doesn't operate the way they thought it would, or it's too slow, or they don't like the graphics or almost anything else. In most large systems, the time between identifying the high level requirements (in the ConOps) and presenting the final system, can be months if not years. By then, the client's need (business requirements) have either changed or become better understood, driving the need for additional or modified requirements. These changes are usually implemented in a follow-on release, much to the client's chagrin.
Of course, the needs can be addressed in the current release, but usually at a significant cost in re-engineering, re-coding, re-testing, documentation, and, depending on how late in the processes the changes are made, training. It will also inevitably take more time. In fact, as the diagram below demonstrates, the later in the release changes are made to the requirements, the greater the risk to the project, either in time, money, or both.
Agile Requirements Development
Agile requirements development uses principles similar to Agile software development. Those principles include:
- Dedicated Resources - Personnel dedicated to the completion of the project and this phase of the project.
- Collaboration - the Client and the Agile team members work together closely to develop the User Stories (no "the system shall..." statements).
- Brief team meetings to discuss the progress or problems in user story development and clarifying the contents.
- Frequent Feedback - Continuous client involvement ensures user stories are understandable and reflect the user's needs.
- Specifications - If needed, the development of specification addressing some details of a user story (e.g. how would an application accomplish the tallying of elements of multiple columns, or what factors does an application have to use in order to accurately select values to tally).
Collaboration
Ideally, Agile requirements are in the form of User Stories rather than "the system shall..." statements. And as such, they may create some ambiguity with respect to what they solution to the user story should do.
Within an Agile environment, requirements should be developed in a manner similar to the overall development of an application's functions. The client doesn't have to define the application down to the very last function. Likewise, the client doesn't have to have a complete set of user stories. One of the primary principles of Agile development is collaboration.
Rather than developing the requirements in a "Developmental Vacuum" in which the analysts are isolated from the designers and developers, and the testers are separated from the users and analysts, the entire team works out the user stories. The entire team works out the ambiguities (for instance there may be an order in which tallies are accomplished, or there may be a logical state that has to be explained).
This process also helps the team to avoid surprises while developing or testing. The process ensures the testers test for what the developers coded and that the user expected.
Close collaboration generally leads to valid user stories understood by all participants in the project. This understanding, from purposeful collaboration, can reduce development time, testing time, Independent Validation and Verification time. Each of these savings equates to a financial savings.
Daily Scrum
What environment can best allow the team to collaborate during the project requirements development? A Scrum, a daily scrum. As with the actual coding of the system, requirements development can be assisted by a daily Scrum.
Within the scheduled scrum the users, the developers, the analysts and Scrum Master should review new user stories and, if there is the least bit of ambiguity or doubt as to the meaning of a user story, brief work sessions can be scheduled to clarify them or to create specifications that spell out some of the expected processes.
By taking this additional step, the user can have greater assurance the developers are understanding exactly what they want done, be it counting cases closed or other parameters for including a case for a report.
As with the development phase of the project, the Scrums should be short, not more than fifteen minutes, and should never be used to develop solutions to problems.
Dedicated Resources
One of the costs of this approach to user story development is the need for a set of dedicated team members. There are items only the user, or Subject Matter Expert (SME) can answer. Where the user story depends on policy, the User must be able to provide the policy and how it impacts the functions. If it is a matter of operations and formerly manual processes, the user must be able to provide guidance.
If that user is not available, the team is not able to move forward with the user stories. Incorrect user stories, or user stories with ambiguities will, potentially, cause the testers to build incorrect test cases and the developers to code incorrectly.
Likewise, each of the other members must be available for the scrum and for whatever clarification or validation sessions that may take place. They should be there both as receivers of information (solutions to the ambiguous user stories) and as contributors to the solutions.
At all times, the essential issue is having the people who can make decisions available, whether those team members are analysts, users, developers, or testers.
Feedback
If the user stories are developed in more than a single sprint, as is probable, one of the activities included in the sprint is a sprint retrospective. What went well? What didn't go so well? What needs improvement?
It's here the team learns to write user stories that are understandable, unambiguous, and testable.
Experienced team members may come to the project with the skills needed to create meaningful user stories. If not, with a properly functioning team, the members are able to develop the skill, not only enabling them to contribute to the team they are currently working with, but making them an asset for the next project.
It is important to note that in order for this feedback session to work, the attendance should be limited to the Scrum Master and the team members. The client should not attend since it might hamper open communications between the team members.
Specifications
For user stories of deceptive simplicity or that require the software to accomplish such things as fairly complex filtering and tallying, the use of specifications can make sure the software is doing what the user thinks it's doing, what the analysts think it's doing, and what the testers (project and IV&V) think it's doing.
The specifications can be as specific as the order elements in a table are added together, what parameters are used to select a file for inclusion in a tally, when to display an image, or what values should be present to delete a file.
Specifications provide the final statement in determining if the software is operating. It should not be confused with a requirement. Additionally, it is not a specification in the sense of say, a design specification. Its purpose, as stated previously, is to fill out the user story, to dispel any ambiguities.
The Advantages
An Agile approach to requirements development has several advantages. The involvement of the user or client in the development of the user stories by definition keeps the user informed of the progress of the user stories. The involvement of the developers and testers keeps them informed as well, minimizing any surprises during development or testing[2].
Developing user stories in sprints permits the most effective collaboration between the team members. This time is also spent ensuring the meanings of the user stories are accurately captured. Sprint time can also be used to develop user story specifications, explanations of what they software will accomplish and what the results will look like. The involvement of all team members improves the likelihood that all the team members will understand the function the same way.
References
[1] http://www.dtic.mil/ndia/2010systemengr/WednesdayTrack1_11106Matuzik.pdf
[2] Read, Aaron Describing Agile Requirements Development and Communication using Complex Adaptive Systems Theory 2012.