CONSTRAINTS AND CONCERNS IN DATA
COLLECTION AND ANALYSIS
Often it is simply not possible to gather all the information needed, especially in crisis and unstable contexts.
Time, access/security, resources and the emotional impact of the crisis should all be taken into consideration
when defining the information needs, as they could impact data availability and/or usefulness.
Awareness of those constraints means:


Seeking alternatives (collection of data through proxy indicators, different combination of techniques,
different data collection strategies, etc.)
Knowing the limitations on the quality of available information (e.g. lack of coverage) and therefore
potential bias in the results. Those limitations should be carefully interpreted and presented.
The following is an illustrative (not exhaustive) listing of constraints and concerns in data collection and
analysis.
FLUIDITY OF CONTEXT
The problem
Rapid change
of situation
In emergencies and complex
emergencies, the situation changes
frequently. The complexity of this
volatile context is difficult to capture.
Crises can change rapidly; they are
volatile and dynamic.
Different
timelines
Regional
variations
The development of a crisis can be
different even in contiguous areas.
There could be considerable variation
at regional level.
Complexity
Implications/adaptations for data
collection
 Focus on information relevant in a changing situation.
 Assessment should be planned with future monitoring
in mind (build on successive data collection
activities).
 It might mean needing to set early warning systems
into another area.
 Selection of samples and coverage should consider
the need to capture regional variations. It should be
kept in mind that findings may not be generalisable,
and applicable to every context.
LIMITATIONS IN EXISTING DATA AND DATA COLLECTION SYSTEMS
Absence of
baseline
information
Difficulties to
construct
timelines
Gaps and
problem in
data sets
The problem
Implications/adaptations for data collection
Frequent absence of baseline data on
the condition of the affected population
prior to the assistance intervention or
changes in context make original
baseline obsolete.
The chaotic nature of crisis and
unstable contexts means that the
precise sequence of events in the
period can be difficult to trace.
 Innovative use of methods to fill in gaps is necessary;
for example, using RAP techniques to trace the
evolution of the situation.
Gaps in data coverage (e.g. no national
coverage), unavailability of
disaggregated data (e.g. by gender,
age, ethnical group), non-standard
format of data.
UNICEF M&E Training Resource
 Data collection will need to focus on this,
brainstorming with different groups of stakeholders as
well as validating carefully.
 Telephone interviews with staff who have moved on
may be necessary to fill in gaps on early phases of a
crisis.
 Careful analysis is required of what data exists, with
an effort to define the universe represented.
Capturing a national or regional picture often requires
careful extrapolation and interpretation from data that
is available and/or new data.
 New data collection can provide more in-depth data
sets based on purposive sampling.
Constraints and concerns 1/5
Absence of
national/ local
partners
Weak
monitoring
systems
Civil society and government
administrative structures can be
disrupted, meaning an important
partner for information exchange will be
missing.
Complex emergencies and disaster can
result in the absence of these likely
counterparts or they can be very weak
among the many international
stakeholders.
Disruptions and physical destruction of
facilities often lead to gaps in regular
programme monitoring, information
systems and archiving.
 Wherever possible, new data collection activities
should include capacity building activities for
national/local information services.
 Enhancing information sharing with other actors
whenever possible and opportune is important.
 This raises issues for managing a stakeholder
evaluation process as well as for management and
resource consideration. Also, this increases the
pressure on and the significance of the evaluation’s
accountability role.
 More new data collection is often required where
monitoring systems are stronger.
 For monitoring, substitute activities such as more
frequent and structured field visits are critical.
 Ensure monitoring of key indicators, effective filing
systems, standardised monitoring procedures
 Ensuring effective debriefing: it can be difficult to
capture the experiences of all key personnel that
have been involved in the projects once they have
gone. The reasons for particular choices, the
implications of some events are likely to be lost.
EMOTIONAL IMPACT OF CRISIS
Trauma of
affected
populations
Change
Effects of
stress on
personnel
The problem
Implications/adaptations for data collection
"The affected populations will have just
experienced a civil war during which
they may well have been forcibly
displaced, had relatives and friends
killed, either in the fighting or during
atrocities committed against civilians,
and seen their personal, social and
perhaps cultural identities shattered".
(OECD)
Similarly, natural disaster will deeply
distress the affected population.
To what extent old data and
perceptions can be applied to the new
situation?
New actors will appear after the conflict:
understanding the new stakeholder
context is important. Who are the new
actors? How do we deal with them? Are
reliable? What is their stake?
Changes in political players/ fighting
factions/alliances
Stress can impact personnel during the
crisis or later on. In the evaluation
phase, for example, fatigue of
personnel could result in their
unwillingness of recall their
experiences. National staff will
experience the stresses differently and
will have added concerns about their
own families.
Data gatherers may be affected by
trauma. Psychological trauma of seeing
bodies or wounded people combined
with the fatigue common in this
situation can be a serious problem for
objective data collection.
In stressful and demanding crisis times,
teams are likely to experience tensions,
conflictual views.
 Awareness and capacity of dealing with trauma is
important when performing data collection and
analysis activities.
 Use techniques where data gatherers have
opportunity to manifest care, support. Use discretion
— don’t be intrusive.
UNICEF M&E Training Resource
 Need not to stereotype, but understand new roles and
new behaviours (e.g. through VCA/Stakeholder
analysis). Be aware that civil wars greatly exacerbate
and deepen existing fault lines within a society and
may well create new cleavages in previously coherent
groups (OECD. P 11).
 Managers of data-collecting activities should organise
regular debriefings with data gatherers to gauge their
stress level. Be ready to withdraw staff if necessary.
Stress counselling should be provided if needed.
Constraints and concerns 2/5
CNN effect
Humanitarian programmes with their
focus on saving lives in extreme
situations are often accorded a high
profile by media and politicians. The
resulting increase in sensitivities, to
direct or implied criticism of
programmes, impacts both on the
process and how results are places in
the public domain (source: Alnap)
 Ensure that data collection is not focusing only on the
“issues in the spotlight”. Increase your efforts to
produce quality, non-biased, impartial data.
TIME
The problem
Implications/adaptations for data collection
Urgency
In a crisis, time is a key.
Heavy
demands
In a crisis, the demands for information
are more than what can be managed.
Conflicting
priorities
The pressure on personnel means that
they can have no time to be involved in
the data collection process even when
they would need / like to do so.
 Prioritise information needs, and collect only what is
needed.
 Rapid techniques for data gathering (e.g. RAP)
should be employed.
 Urgency will be a factor in determining the level of
precision of an indicator.
 Need for careful co-ordination among programmes.
Ensure integrated planning of assessment activities
(IMEP).
Minimise the need for new information by using existing
sources
 When required, staff time should be explicitly
allocated for data collection, dissemination, exchange
(e.g. for internal monitoring and evaluation)
SECURITY, ACCESSIBILITY
Sensitivity
Access to
areas/people
Marginalisation
The problem
Implications/adaptations for data collection
Perceived sensitivity of data can cause
security problems in assessments.
Even basic data and information
needed in a crisis (e.g. a map) could be
considered sensitive data and not be
available. Asking the "wrong” questions
could mean being perceived as a spy.
The access to certain areas/groups of
people could be restricted by physical
factors (e.g. destruction of road,
seasonal factors) and security menaces
(e.g. roadblocks, threats from armed
factions).
Access constraints could make
impossible to use formal sampling
methods.
 Be aware of political sensitivity of information
 measure the risk of "asking the wrong questions".
 Combine different techniques (e.g. use observation)
to probe and collect sensitive data indirectly.
 Strive for objectivity and show impartiality. Ensure
that all key stakeholders (including groups with
conflicting interests) are represented
 The logistic of data gathering information is
particularly important in crisis situations.
 Techniques such as rapid field assessment can
quickly gather basic data about areas affected by
security or access constraints.
 Remote control systems/networks of informants can
at times be the only way to get information from
inaccessible areas.
 In extreme, the two-hour visit can provide information
about areas that could not be accessed otherwise (
Rapid field visit).
 Data gathering operations should provide information
disaggregated by sex/gender whenever possible.
 Gender analysis must be incorporated in crisis
assessments.
Crisis situations can exacerbate the
"invisibility" of certain groups (on the
basis of gender, age, ethnic origin,
religion, etc.)
Failure to recognise relevant
stakeholders can end in biased data.
UNICEF M&E Training Resource
Constraints and concerns 3/5
Partiality
Polarisation
Despite the fact that security and
accessibility can undermine the quality
of the data collection, data collection
and analysis cannot not wait until after
the crisis is ‘over’ or until the context is
stable. This mean that only partial
results can sometime be achieved.
Complex emergencies in particular are
characterised by high degrees of
polarisation and politicisation, it is
difficult to handle the diverging
perspectives among stakeholders.
 Recognise the need for sensitivity and flexibility in the
data collection and analysis process.
 Point out possible limitation in the quality of data.
 Strive for objectivity and show impartiality. Ensure
that all key stakeholders (including groups with
conflicting interests) are represented.
COORDINATION AND INTERDEPENDENCE
Attribution of
impact
Multidisiplinarity
Co-ordination
The problem
Implications/adaptations for data collection
The emphasis on co-ordination in crisis
and unstable contexts means that
programmes are very inter-related and
a broader perspective must often be
taken to get a more comprehensive
picture of programme effectiveness. In
general, it is more difficult than normal
to demonstrate plausible programme
outcomes or impact (i.e. to differentiate
the effect of crisis and of concurrent
programmes).
Humanitarian intervention is often
highly multidisciplinary – e.g. public
health, logistic, food and nutrition, water
and sanitation etc.
In times of crisis, especially in the
onset, co-ordination can be difficult or
absent.

An attitude of knowledge sharing amongst different
organisations and team should be nourished.
 Teams are required to reflect such diversity in their
composition and expertise.
 Co-ordination efforts among different programmes
are needed (e.g. integrated plans) for data sharing.
RESOURCES
The problem
Implications / adaptations for data collection
Financial
Data collection is costly and can
be even more so in times of
crisis (e.g.: need of extreme
rapidity, cost of operating in
regions where infrastructure
collapsed)
Short-term funding leads to
dangerous short-term
perspective on monitoring
trends.
Personnel
 Working overload
 Lack of personnel
 Turnover
Logistical
resources
Collecting information means
using vehicles and logistical
resources that are already in
high demand to implement the
programme.
 Evaluating the cost of obtaining information is a key factor in
choosing the information needed.
 The cost of the information should be considered in terms of
how long this information will be valid in a changing context.
 In some cases some information (e.g. requiring detailed
surveys) might not be obtainable with the financial/human
resources available.
 Wherever possible, data collection exercises should seek to
cover information needs for various programmes on a wide
range of issues ( IMEP)
 Collaboration with other organisations is important.
 Data gathering operations should take into consideration the
personnel workload.
 Be aware that by being immersed in the crisis, personnel
can lose perspective, e.g.: miss signals that should trigger
action. It may be necessary to combine staff new to a
specific area with those who know the area.
 Need to build assessment and monitoring into “everyday
life”, by using meetings and routine visits to gather data.
Also, non-expert staff (e.g. drivers) in the course of their
duties could use the checklist for observation.
 Co-ordination with other organisations means being able to
use data you didn’t have to collect.
UNICEF M&E Training Resource
Constraints and concerns 4/5
Skills
A staff needs different skills for
conducting assessments and
evaluations in crisis and
unstable situations.
Be aware that assessments tend
to involve more interpretation
and analysis at field level than
traditional centralised
monitoring.
UNICEF M&E Training Resource
 Staff participating in training programmes could enhance
their skills. (Consider also employing self-learning tools.)
 Establish a roster: The availability of a list of experts could
help identify and quickly contact the most suited
professional. This should be done with a roster of locally
available personnel. For international personnel, consider
also contacting non-commercial organisations, providing
human resources for operational humanitarian agencies
(e.g. RedR, International Health Exchange). See
http://www.redr.org/resources/links/hr.htm.
 Consider providing skills to non-expert staff so they can
contribute to data gathering (see above example of checklist
for drivers).
Constraints and concerns 5/5