Draft EMOPS 2009-03-17
QUICK GUIDE TO DESIGNING AN AFTER ACTION REVIEW
What is an After Action Review?
An After-Action Review (AAR) is a structured ‘lessons learned’ meeting with key stakeholders to
review a project or action, probing what happened, why it happened, what was learned and how it can
be done better next time. The practice originated with the United States military but has long been
adopted and adapted much more widely and increasingly so by organizations in the humanitarian
system.
The “After-Action” reference is a misnomer as AAR has been applied early in a humanitarian
response. It is clearly the lightest most widely applicable form of review exercise and could be applied
periodically through an emergency.
When to use AAR in humanitarian contexts?
 For small scale emergency response where the relatively small size of the total budget does not
justify a more in-depth evaluative exercise;
 When a planned review or evaluation exercise will be delayed until late in the response; the AAR
then serves as an intermediate exercise and as an input to the review or evaluation.
 As an ongoing quick review, the AAR can be used periodically during a response.
If the AAR is delayed too long after the phase or event to be assessed, it is better conducted as part
of a wider Lessons Learned exercise with a documentary review as a key input to the workshop or
meeting.
What is the objective of the AAR?
 Within the limits of the methodology, the AAR gives a quick analysis of effectiveness of response –
in particular highlighting the strengths and weaknesses of the actual as compared to the planned
response.
 It contributes to learning by those involved to strengthen future humanitarian action – it probes
why strengths and weaknesses occurred, and draws out lessons on what to preserve or do
differently next time.
 Depending on the timing of the AAR, it can contribute to improvements for ongoing response –
it can yield immediate action points for key stakeholders.
Limitations
 AAR on its own is a quick qualitative analysis. It can be strengthened by having quantitative
analysis done prior to the exercise and share with participants to help focus the issues to be
analyzed.
 AAR necessarily has a limited participation – only so many people can effectively discuss and
analyse issues in a participatory review meeting. This limitation can be balanced somewhat by
wider consultation defining the scope or wider presentation of results for validation afterwards. An
AAR process can also be decentralized or done in series in different fora. (See more below.)
 While participatory, the AAR is not a mechanism by which affected populations can be involved.
Some of the same analytical questions could be translated for community meetings or focus groups
in a wider impact monitoring and evaluation process, but this falls well outside the scope of the AAR.
Defining the scope
It is important to frame the scope of analysis in terms of:
 Timeframe -- what phase(s) of the response;
 Organizational scope – one organization alone (its country office or also including regional and
headquarters offices); a group of key partners; or the wider Humanitarian Country Team;
 Sectoral scope – whether a selection of key sectors or cross-sectoral; it is important to consider to
what degree it is important to look in-depth in each sector;
 Specific themes – are there any gaps or positive lessons easily identified before the workshop that
should then shape the design.
Who should be involved?
In general, those with critical responsibility for key areas of the response should be involved. The list
will vary according to the scope of the exercise; i.e. what do you want to review?
1
Draft EMOPS 2009-03-17
If the AAR exercise is taken to an inter-agency level, it may be necessary to break the exercise down,
doing the review with separate smaller groupings, using for example cluster/sector coordination fora.
Such a series of AAR meetings or parallel group work can then be brought together in an overall
analysis with a cross-sectoral management group.
Wherever there are tensions among participants or perceived power barriers it is important to bring in
a neutral facilitator. If it is a small exercise with a cohesive group, a participant can be named to
chair.
Preparation
 Stakeholder understanding of AAR methodology and its limitations – Because the AAR methodology
is light and largely qualitative, it is important that all stakeholders involved understand the proposed
methodology and its limitations.
 Consultation on scope/issues -- In defining the scope of the AAR, it may be useful to sound out a
wider group of stakeholders to help define specific sectors or issues for focus. Wider consultation
can also be used for a first brainstorming on key strengths/weaknesses – this can help reduce bias
in defining the focus of the exercise. Such consultation can be done through pre-meetings and/or a
survey. This becomes important where the scope of the AAR is larger, e.g. at Inter-Agency level,
and the methodology necessarily restricts participation.
 Management group – A small group of staff (3-4) needs to be assigned to work with the facilitator in
developing the process design. It is important to use representation on this group as much as
possible to build buy-in of key stakeholders and give voice to different groups.
 Documentation – Agree on a simple means of documentation -- some form of final report is
important for accountability and for sharing the lessons learned beyond participants. The report can
be short but should include: key strengths, the contributing factors and ultimate lessons; key
weaknesses/gaps in performances, the contributing factors and lessons; key prioritized
recommendations and/or follow-up points. Annexes should include: an outline of the process, the
agenda and participants list.
 Facilities – The usual workshop considerations apply – consider lots of wall space as the analysis is
accumulative and kept posted on walls to help participants recall and visualize earlier discussions.
Workshop process design
The following is an outline of considerations following a basic process of 6 steps, assuming a single
country exercise reviewing a general cross-sectoral response. Obviously, the design has to be
adapted to the specific case, considering who participants are and the scope of the exercise.
Examples of how to use techniques from Visualization in Participatory Planning (VIPP) 1 are included
as these are useful to quickly draw out participants’ thinking and facilitate a common analysis. They
can also help to encourage better participation, especially in a context where sensitivities and power
relations may create barriers to direct contributions by all participants. Key practical details for those
unfamiliar with VIPP are included in Annex A.
Introduction
 As with any workshop, the facilitator should: set ground rules, review/validate the agenda, ensure
that participants know each other.
 It is also important to make it clear to participants how workshop outputs will be used; where
recommendations will be tabled and what is the process for establishing follow-up.
 For larger AAR, consider the value in a slightly more formal opening where a key leadership figure
sets the tone: emphasizing the value of the exercise and the need for open analytical and
constructive discussion and committing to using the results.
Analysis of results vis-à-vis plan
 For this phase, it can be useful to post on the wall on large flip chart pages the objectives of the
phase or part of the response to be assessed, extracting these from the established response plan –
e.g. from the latest Appeal document. This provides a reference for discussion.
1
UNICEF (1993). Visualization in Participatory Planning available on UNICEF Intranet. This manual is a synthesis of methods
for improving group interaction in international cooperation programmes. The approach is widely used in UNICEF since the
1990s.
2
Draft EMOPS 2009-03-17
 The key question to guide the discussion in this first stage of analysis is “How did the response
actually play out? What were the biggest strengths and weaknesses in the response?”
With VIPP, this can be done as a quick brainstorm, inviting each participant to write up and
post 2 cards each on strengths and weaknesses (different colour cards, same shape). The
facilitator and a few volunteer participants then cluster cards, showing convergence of
common issues, using large circular cards to give titles to closely linked issues. Then the
facilitator leads participants to do a dot-voting to prioritize the issues considered most
important, be they strengths or weaknesses (three votes per participant) – this last is in
preparation for the next session.
 Note that for this step in the analysis, it is important to consider whether the discussion on
strengths/weaknesses is left open or structured by providing themes. For example, a UNICEF
discussion might be structured by already posting on the wall a series of headings corresponding to
each of the Core Commitments for Children. (This also helps in a first round of clustering issues.)
Different levels of structuring can be considered; for example, simply proposing themes as a prompt
to trigger wider analysis by participants, or specifically tasking participants with identifying key
strengths and weaknesses for each theme.
 If an AAR has been defined to address specific sectors or themes, a more structured approach will
be appropriate.
 It is important to prioritize in this process – too many strengths/weaknesses identified without
prioritization will ultimately lead to too many recommendations.
Analysis of strengths
 The questions that guide the discussion in this second phase are: for each of the strengths
identified, what factors made this possible and what are the lessons to be drawn for the future, in
terms of how replicate the success. If using VIPP, it is possible to merge this with the next phase.
Analysis of weaknesses/gaps
 The questions that guide the discussion in this phase are: for each of the weaknesses, what factors
contributed to this and how can things be done differently in the future.
 Where participants are numerous, it may be better to divide up the issues for analysis among
smaller groups, each group tasked with analysing strengths and weaknesses around a specific
issue or grouping of issues. If this is done, it is important that there is a mechanism whereby all the
relevant participants can understand and add to the analysis. Options include: rotating groups to
review analysis of the previous group with the opportunity to add to the analysis; standard
presentation by groups to plenary with opportunity to discuss; presentation to plenary by each group
with other groups tasked with critiquing the work from pre-assigned perspectives (e.g. a group could
be assigned to listen for analysis of CO-RO-HQ roles).
Using VIPP, for each prioritized strength or weakness, have participants describe to the group
contributing factors and provide any concrete examples. As participants speak to the group,
make sure the is common understanding of what is said and then facilitator, an assistant or a
participant is tasked with writing these up on cards (cards of corresponding colours in different
shapes for contributing factors and examples). Contributing to a given strength, there may
be some constraining factors too – nothing is completely positive or negative.
Strength
factor
factor
Eg.
factor
factor
factor
weakness
factor
factor
factor
Eg.
For each strength/weakness, lead participants to phrase a lesson (conclusion on more
generally replicable good practice or need for wider organizational systems change) writing
this up on a flip chart.
3
Draft EMOPS 2009-03-17
For write up, this VIPP method converts easily to a format for a note-taker while the session is
ongoing. This can then be quickly reviewed by the facilitator who can elaborate the text more
from memory to avoid the usual problem of VIPP cards only being understandable to
participants.
S/W
• Strength
Contributing factors
• contributing factor
• contributing factor
• contributing factor
Examples
• example
Recommendations
 Note that the next step after a good analysis of a problem/success and the lesson, is formulation of
a recommendation. A good recommendation should be specific, clear, actionable and achievable.
This is often difficult and can benefit from multiple stages of review.
 It is important to note that the forum for identifying lessons learned is not always the best forum for
defining recommendations. For example, in a CO, the forum for defining recommendations might be
the CMT or a special task group reporting to the CMT. It requires involvement of those responsible
for the action as well as mediation by someone who can challenge stakeholders to fully address the
issue identified, i.e. to avoid recommendations that do not address the heart of the problem. When
it is a question of a global systems change, a good recommendation often requires inputs from CORO-HQ perspectives for it to be actionable. This is important in defining what happens with the
output from the meeting – should it be taken to another forum for discussion, validation and then
setting up a follow-up mechanism.
 It is important to prioritize recommendations. The facilitator must lead participants to identify the top
priority lessons to be translated into recommendations.
 Where participants are more than 10-12, it is useful to break participants into groups to work on
recommendations and task groups with formulating recommendations according to a structure: who
is responsible, who else should be involved, what is the timeline, what is the action required, what is
the expected result. (The latter pushes people to consider whether recommendations actually
address the problem identified.)
Summary and next steps
 As with any workshop or meeting, it is important to close with a clear quick summary of what has
been covered and what key points have been identified as well as reach clear agreement on
where/how results will be used and the follow-up mechanism.
4
Draft EMOPS 2009-03-17
ANNEX A: PRACTICAL DETAILS ON VIPP




Cards are
Clustering cards
Dot voting
Supplies needed
o Cards – basic supply is 4 different colours (light colours so marker is legible), each in 4
shapes (usually rectangle, oval, large circle, small circle); rectangle cards are about 20cm
x 15cm, oval comparable dimensions
o VIPP boards – the best are large 1m x 3m foam boards that can be propped up on a flip
chart stand and easily moved around; alternative options are large rolls (1m wide) of
brown paper that can be taped up on the wall
o Depending on the type of board and supplies available, something to easily attach cards
to the board – pushpins; spraying the board with 3M spray glue; tape
5