Emphasizing Cultural Competence in Evaluation: A
Process Oriented Approach
Authors: Luba Botcheva, Johanna Smith, Lynne C. Huffman
American Journal of Evaluation June 2009
Naina Limbekar
Cultural Competent Evalution:
Authors Luba Botcheva, Johanna Shih, and Lynne Huffman discuss the importance of conducting culturally
competent evaluations and describe the process-oriented approach and specific strategies they used when evaluating an
HIV/AIDS education program in Bulawayo, Zimbabwe. Cultural competence in evaluation can be defined as the design of
“appropriate programs, standards, interventions, and measures so that they are specific, relevant, and valid for each
unique group” (SenGupta, Hopson, & Thomson-Robinson, 2004, pg. 13). Essentially, it is important for evaluators to
adjust for differences in viewpoints between themselves and users of an evaluation by adapting the fundamental steps of
an evaluation: definition of successful outcomes, selection of methods and instruments, collection and analysis of data,
and dissemination of evaluation results. Cultural competence in evaluation serves not only to ensure social justice to the
population being served but also is essential for improved data validity and evaluation results.
Process-Oriented Evaluation:
Most commonly, evaluators and the populations they serve do not share the same culture. In order to address this
issue to conducting culturally competent evaluations, the authors suggest that three important components of a processoriented approach should be considered. The component of collaboration involves that evaluators collaborate with
stakeholders to gain knowledge about the cultural aspects of a paricular population and this knowledge can thus be used
to identify goals, methods, and measures that are the most relevant to the program. Secondly, the component of reflective
adaptation requires that evaluators readily acknowledge ones’ biases toward any aspects of a population or a setting and
understand what roles these biases play in the evaluation process. Finally, the last component to the process-oriented
approach is that of contextual analysis. Contextual analysis requires that an evaluator pinpoint cultural views as it fits into
an appropriate societal context; evaluators should observe how certain beliefs and norms are created in reaction to and
interaction with the surrounding environment. Moreover, the authors emphasize their view of cultural competence as of
that on a continuum versus that of categories (culturally competent or not). This view will allow evaluators to focus their
efforts in achieving the highest level of cultural competency in a given evaluation.
A Process-Oriented Approach to Evaluation Grassroot Soccer:
The authors reference their evaluation of the Grassroot Soccer HIV/AIDS Prevention Program (GRSF) to
showcase how they implemented the process-oriented approach to conduct a culturally competent evaluation. GRSF is a
program that involves training adult soccer players (powerful role models in Zimbabwe) to utilize HIV/AIDS life skills in the
education of at risk preteens to increase knowledge and improve attitudes about their ability to prevent the disease. The
main focus of the evaluation was to create an evaluation instrument that was culturally appropriate. The existing survey
used to measure program changes on control and intervention students consisted of multiple choice questions which had
yielded contradictory data from the students because they did not understand the questions. The evaluators completely
modified this to create a survey that was comprised of simple true and false statements, vignettes with imaginary
characters students could relate to, and replaced Western concepts of “choice” with Zimbabwean of “right v. wrong”.
Also, the evaluators used poems that students wrote in order to assess reoccurring emotions, themes, and perceptions
about HIV/AIDS. Since the evaluators were not in the field in Zimbabwe during the evaluation, they used collaboration
with staff to learn about Zimbabwean culture and the realities of everyday preteen life. Also, they employed reflective
adaptation when modifying the survey format and including analysis of students’ poems based on the fact that the
Zimbabwean culture is very narrative and emphasizes different concepts. Finally, the method of contextual analysis was
reflected in the fact that the team of evaluators not only considered the national culture but the youth preteen culture and
how it can be influenced by different environmental factors such as poverty and peer pressure. Nevertheless, it is
important to note that real world limitations (i.e. funding, time, etc…) exist that may impede in performing culturally
competent evaluations.
References
Botcheva, Luba. Shih, Johanna. Huffman, Lynne C. “ Evaluating Cultural Competence in Evaluation: A Process-Oriented
Approach”. American Journal of Evaluation, Vol. 30, No. 2, 176-188 (2009).
SenGupta, S., Hopson, R., & Thompson-Robinson, M. (2004). Cultural competence in evaluation: An overview. New
Directions for Evaluation, 102, 15.
WRITTEN DOCUMENT REVIEW AND GRADING
Group/Person:
Title
Category
Mechanics
Item
Surface writing
errors
Grammar
Language
Citations
Structure
Sentence
Paragraph
Paper
Transitions
Support
Thoroughness
Appropriateness
Connection
between support
and
recommendation
Balance
Perspicacity
Concise
Make points?
Explicit
Clarity
Aesthetics
Graphics
Use of space
Comments
Category
Item
Overall Readability
Flow
Comments
Logic
Compliance
Formats and
protocols
See comments within the paper
Overall assessment: