PLAN REQUIREMENTS SOLUTION ANALYSIS DESIGN BUILD TEST TRAIN/DEPLOY MAINTENANCE TO THE DOCUMENT OWNER: This template is provided as a guideline and resource. The structure and instructions give detail about what might go into a completed document. Only you and your team, however, know what will best fit the needs of your specific effort, and you are encouraged to adapt the template as appropriate to meet those needs. Lessons Learned Guidelines Overview A lessons learned review, or retrospective, should be held at the end of all projects and major service outages. During longer projects, it may also be helpful to hold lessons learned reviews after specific milestones, project phases, or challenges. The overall purpose of the lessons learned process is to review an effort or event in order to pinpoint its successes and failures specifically with a focus on increasing success on future efforts. With this in mind, a project lead or application owner can identify appropriate times for conducting lessons learned reviews both during planning and as efforts unfold. Process The lessons learned process should include the following steps: 1. Hold one or more lessons learned discussions with affected parties. 2. Document feedback from the discussion(s) and identify action items. 3. Assign the action items and follow up to ensure all are carried out. Each of the steps will be described further below. The identification and completion of action items is a key element to improving processes within an organization. Without this work, valuable feedback may be lost and the effectiveness of lessons learned reviews may be questioned by participants. Lessons Learned Discussion The lessons learned discussion is the part of the retrospective process which is most widely familiar. This type of discussion may go by a variety of names, including post mortem, plus/delta, and feedback or whining session. The focus of the discussion, however, remains the same: Encourage participants in the effort to provide specific feedback on both what worked well in the effort and on what could be improved. Several guiding principles, below, help create a productive environment for the discussion. Hold the review shortly after the effort is completed Page 1 of 4 o A lessons learned discussion should occur no more than 2 weeks after the end of a project, the occurrence of an outage or other event, etc. Memory of the details associated with a specific event or effort is quickly diminished by other day-to-day activities. It is important to engage the participants soon after the event in order to obtain the best feedback. This is also an argument for reviewing lessons from the early stages of a project separately from the review at the end of the project; by the project's end, recollection of the planning and requirements processes, for example, may be less distinct and/or overshadowed by later work on the project. Engage someone outside of the effort to lead the discussion o This provides two primary benefits. The first is that everyone involved in the event has the opportunity to offer feedback; doing so can be difficult or distracting while also recording input from the rest of the team. The second is encouragement of all perspectives. When someone who was involved in the effort leads the discussion and/or transcribes the feedback, that person may impose his/her own views of the event on the feedback, which can result in unintended interpretations, phrasing, and even omission of feedback. A more problematic situation may also occur if the facilitator responds to or defends feedback from the team rather than acknowledging and recording it. Provide a safe space for feedback o Many of the most important lessons can be gained from critical feedback, and participants in the discussion should be encouraged to share thoughts on how the effort was unsuccessful as well as on how it went smoothly. The facilitator should begin the session by setting ground rules, which include guidelines such as no personal attacks, openness to all opinions and perspectives, and agreement to share thoughts openly in the session paired with limiting certain feedback to the confines of the group. This sort of preparation makes it easier for those involved to speak freely. Gather feedback from multiple perspectives o While it is traditional practice to gather feedback from the members of a project team, it can be equally useful to collect input from an effort's customers or those who were impacted by the effort, as well as from other groups who may have been more loosely involved in the effort. These people can have a great influence on the success of a project or other effort, and they will often have a different perspective to offer. Including them in the lessons learned effort may provide important information. At the same time, it may be best to gather feedback from these groups separately from the project team session or even to use a different method for input. The details of who should be included and how their feedback should be elicited may be determined on a case by case basis. Provide specific questions for participants o The purpose of a lessons learned session is not to allow those involved in the effort to gripe but rather to identify the root cause of elements that went well and that went poorly. Specific questions may help to guide participants' input and lead the group toward more effective feedback. Some suggestions for questions include: What parts of the project/outage/event went well or were successful? Lessons Learned Guidelines Page 2 of 4 What parts of the project/outage/event did not go smoothly or as planned? What would you do differently in a future effort? What did you learn from the project/outage/event? Is there anything about the project/outage/event which puzzles you? Allow adequate time for discussion and plan a schedule o A group usually needs about an hour to share its thoughts on successes and failures. More time may be required for a particularly large, complicated, or disastrous effort, in which case it may be wise to divide the feedback sessions into smaller parts rather than to schedule a single, long session. If the group is to continue to the next step in the process, the prioritization of issues or the identification of specific action items, this may also require a longer session. To ensure that the discussion not linger too long on a particular topic, it is helpful for the facilitator to have a rough outline of the discussion in mind, with times assigned so that s/he may move the group along at a pace that will allow time for all questions. Document and share all comments from the discussion o The comments from the feedback session(s) should be documented and shared with the participant group and, If there are no objections to a wider audience, it may also be published to a larger group. This documentation should capture the individual suggestions and comments, so that they may be referred to during the next steps of the process and so that all participants' input is represented. While specific comments are documented, who provided each comment should not be included. Action Item Identification The second part of the process involves reviewing the raw feedback to identify specific actions that can be taken. These actions can include both steps to ensure that successful work continues and steps to make changes where needed. Identification may involve the same group which provided the original feedback -- in which case, identification is usually included as part of the feedback session -- or it may be performed completely separately, or it may be some combination of the two. The key to this work is defining specific, actionable steps which can benefit future efforts. Some recommendations for this part of the process are: Identify themes and trends o Feedback comments will often cluster around certain themes or topics, which tends to indicate more significant problem areas. Categorizing or grouping the comments may help with the identification of root causes or of broader issues, as well as with prioritizing future actions. Focus on the future o Feedback is expected to focus on the effort in the past, so the creation of action items often involves generalizing the input and translating it into something which can be done in the future. Reviewing the feedback with the future in mind helps to distinguish between circumstances that were particular to the past effort and situations which have a reasonable chance of occurring in the future. The future perspective helps to generalize the issue, which is key in identifying actions that will be useful for other efforts. Lessons Learned Guidelines Page 3 of 4 Involve the service lead or manager o Because feedback items often relate to issues which are best addressed at the managerial level, it is important to involve the manager(s) for the service and/or team members in the review of feedback and identification of action items. A manager may be able to recognize trends which extend beyond the effort as well as to identify higherlevel actions than may be suggested by participants in the effort. Additionally, the inclusion of relevant manager(s) improves accountability and oversight. Document the action items o The list of action items should be distinct from the original feedback document. It consists of specific, future-focused actions ready for assignment. General statements such as "Communicate better" do not qualify as action items, because they cannot be readily acted upon or measured for completion. Rather, choose concrete items such as "Provide formal status reports" or "Review the issues list in steering committee meetings." The action items document should be published at least for the group of people who will be assigned the action items and may be made available to a wider group for accountability and documentation of progress. Action Item Assignment and Follow-up The final step in the process is the completion of the identified action items. Each item must be assigned to a responsible party and a deadline for completion agreed upon. In addition, a manager or other person must track progress on the action items and provide a structure for reporting and accountability. The suggestions below help to facilitate follow-through. Divide the action items among multiple people o The involvement of multiple people increases accountability, divides the effort for quicker follow-through, and reduces the burden upon a single individual. If the full set of action items is left to a single person, it is unlikely to be completed due to conflicting priorities, lack of interest, and lack of assistance when something goes awry. Assignment to a group speeds the completion and thus the impact on future efforts. Schedule regular progress check-ins o Check-in points encourage progress to be made, hold assignees accountable for due dates, and allow discussion of any issues which may arise. Check-ins should be scheduled so long as any items are outstanding. Designate a person to be responsible for ensuring list completion o Selecting someone to be responsible for completion of the items increases accountability and eases the organization of check-ins, progress reports, delegation of assignments, etc. The responsible party also becomes a go-to resource in the face of obstacles to action item completion. This person may also serve as the liaison to management or other groups for providing periodic reports on the follow-up efforts. Lessons Learned Guidelines Page 4 of 4