Reading for the UNHCR Emergency Management Training
Reading 6.3.4. Bias
Assessment information collected from a single source or a single methodology or
technique presents a biased view of the situation.
Health or nutritional assessments, for example, based on samples from individuals attending
a feeding center or clinic will likely be biased because only those people well enough to travel
to the centres will be sampled. Some of the most ill or undernourished may be missed
completely. Therefore the technique will be biased toward those with the energy or ability to
travel to the clinic. The table below provides several examples of likely sources of survey
bias.
Possible Biases of Non-Randomly-Gathered Data in a Refugee Camp
If you sample people:
Bias will cause the results
to appear:
Because
On the streets or in markets
better than actual
ill children are less likely to be
outside
At feeding centres
better than actual
they are getting food; there may
be others who need food but who
are not able to get it
or
In some situations, only worst
cases are allowed into feeding
centres
Worse than actual
At hospitals or health centres
better than actual
worse than actual
only those well enough to reach
the hospital or who have family
who can bring them are
represented
or
Only sick people are going to the
hospital
Near the administrative center
of the camp
better than actual
"wealthier" or more powerful
people may live there
in only one area of the camp
better or worse
people of similar status (and thus
physical condition) tend to live
together
along roads
better than actual
"wealthier" or more powerful
people live there
Adapted from Assessment Manual for Refugee Emergencies, August 1985 designed by
INTERTECT for the Bureau of Refugee Programs, Department of State, Washington DC
Cultural bias is the distorting effect of one’s own cultural norms—results from the team’s
ignorance of the cultural practices of the assessed population. For example, a determination
that the refugee group does not require additional food because food items are being
discarded, may be incorrect if the food items are taboo for cultural or religious reasons.
One way to overcome cultural bias is to include members from the refugee community on the
assessment team, not just as translators, but also as "cultural consultants" to the assessment
mission. Refugee participation (men and women!) should begin as soon as the first groups
arrive. Look for the natural leaders of the refugee community, the elders, elected or appointed
political leaders (such as mayors or other local officials) village chiefs, heads of clans, etc.;
and for groups of refugees that might otherwise not be represented (e.g. women's groups).
Once representatives or leaders have been identified, they can be organised into working
committees.
Gender bias most often results from the lack of women's input into assessment interviews. It
is an important aspect in providing a balanced and useful assessment for the development of
program responses as there is a functional difference between the gender based roles in the
community. Men, for example, might be routinely chosen to receive food distributions for their
1
Reading for the UNHCR Emergency Management Training
families, since they are seen as the heads of households, when in practice and by their past
routine it may have been the women who provided food for the family while men worked for
cash or other non-home based products. To guard against gender bias, both men and
women should be included on the assessment team.
Time of day or schedule bias occurs because an assessment is conducted in a certain
place at a time of day when certain segments of the population may be either over- or underrepresented in the assessment survey. For example, if assessments are conducted at food
distribution centres where men may be the primary food collectors, the survey will not include
women. Similarly, if the survey is primarily implemented in the morning when women (or
school age children) are routinely queuing for water collection, they will be missed.
One of the primary methods of establishing accuracy and avoiding bias is through correlation
of the data—by comparing accounts from different people and different agencies and by
analysing different indicators. Correlation of the data through cross checking independent
sources is a basic element of good assessment practice.
Correlation of information is sometimes called triangulation, the process of intentionally
varying the assessment team’s data gathering techniques and the sources of information in
order to minimise the potential for bias. Data should be evaluated against relevant baseline
data or norms for that area. Remind analysts and decision-makers that assessments may
uncover and highlight pre-existing problems in the host country or area, as well as those
generated by the emergency. Question and double check information that seems
unreasonable (e.g. major variations from already identified trends) and do not be afraid to
follow-up on sources in order to clarify an issue, figures or other details.
In addition, data should be analysed against indicators to look for trends or patterns or to
determine if the data shows an isolated phenomenon. Refer to the "Tools and Resources"
section of this notebook, the Sphere guidelines, and the UNHCR Handbook for Emergencies
for emergency indicators. Compare your findings with these indicators or other typical
patterns found. Cross check data coming from different sources (e.g. different team members,
previous research and reports etc.) The information may have gaps or may be biased. Avoid
generalising from data relating to only one area, one sector or one part of a population.
Assessment team members must also understand the different methods available for carrying
out field assessments and select those that meet the immediate need, as well as those that
can serve to help balance or verify assessment information gathered in other ways.
Direct observation, when implemented correctly, is generally the most trustworthy, explicit
and foolproof assessment method. Observations about the number of people, their condition,
the site, water sources, sanitation needs can all be determined from on-site visual
assessment. Even though "seeing is believing," one should keep in mind the possibility of
bias that may result from only seeing a particular part of an emergency.
In many refugee emergencies the use of aerial or satellite images for initial assessment of
the emergency influx area is extremely helpful. Access to such photographs may be available
from military forces when such units are deployed as part of the humanitarian response to the
emergency. Close coordination with military assessment data can help in establishing
accuracy as well as the general situation of the refugee emergency area.
Two important considerations in the use of aerial or remote sensing images are their direct
usefulness (is interpretation by specialists required?) and difficulty and timeliness of access
(is it worth arguing and waiting for restricted information?) One should bear in mind that
aerial photographs require interpretation which takes time. If the photographs are considered
to be "sensitive" it may not be worth the effort of negotiating access to them if other
assessment options are available.
If fly-overs are to be arranged for the assessment team, careful consideration of weather
conditions, military or security restrictions and all official requests and protocols must be
followed. This is often an expensive proposition and requires the right technical equipment as
well as skilled pilots. In conflict areas, low-level fly overs are a serious security issue and their
advisability can only be determined in context.
Drive throughs or "windshield surveys" are one step closer to the site than fly-overs, yet
may still produce distorted results. Bias may be introduced by limiting observations to those
that can be seen from the road. This view from the road is sometimes called "tarmac bias"
which may be substantial when refugees are Travelling or settling in difficult, inaccessible
areas.
2
Reading for the UNHCR Emergency Management Training
On-site observation is an excellent way to become familiar with a situation. Experienced
observers can gather information quickly if they know what they are looking for. Often.
however, further investigation is needed as some details may be hidden from view. On-site
visual inspection and listening are good methods for an initial assessment. Visit the site and
be attentive and aware of detail. Take notes as you go. The following steps may improve
knowledge and change or dispel assumptions regarding the target population:

Observe people's physical condition and activities and ask questions

Visit homes or shelters, water sources, clinics, distribution centres

Observe the activities/conditions of women (if possible), children, the elderly, and the sick

Observe eating and drinking habits and sources of food
 Observe the services, vehicles, sanitation
Photos, video footage or even hand sketches are extremely useful in communicating to
others the reality of the situation. Photographs (taken with the permission of those you are
photographing and the authorities) are particularly helpful for presenting assessment
information.
Mandate and speciality bias is the natural tendency for
agencies and/or specialists to assess more rigorously those
areas which fall under their competency or mandate. This is
normal and, to some extent efficient, as experts should be
most likely to notice deficiencies within the area of their own
expertise. This can become a problem, however, when an
agency’s mandate and resources lead it to address specific
refugee needs while ignoring other needs which may be
more critical but fall outside that mandate.
MANDATE
NEEDS
RESOURCES
This issue of mandate or speciality bias is best overcome by assessment teams of mixed
specialists and generalists, and or mixed inter-agency or joint assessment teams. This mixing
of viewpoints will have a balancing effect on the weight given to different aspects of the
assessment. Additionally, it will help overcome the resistance associated with sharing
assessment reports between agencies or specialists.
Triangulation also refers to attempts to vary the skills, backgrounds, and demographics of the
assessment team itself to ensure that different perspectives on the same data are available.
By including a variety of specialists and generalists on the team, you should be able to avoid
many of the biases discussed. It is recommended that the team be balanced with "insiders"
who may know the local situation and the customs of the people well but who may also bring
preconceptions and "outsiders" who although less familiar with the situation may be able to
provide the perspective of distance that is also needed in making useful assessments.
3