Donna M. Mertens
Independent Consultant
Minnesota Evaluation Studies Institute, MESI Spring Training
Minneapolis MN
March 2015
 Name, position, organization (no acronyms please).
 Where do you come from?
 What is your experience in evaluation?
 What has been your experience with social justice?
 What do you want to get out of this workshop?
Mertens MESI March 2015
2/19/2015
2
Philosophical Assumptions
Axiology
Respect for cultural norms; support for human rights and
social justice; reciprocity
Ontology
Issues of power & critical interrogation of multiple realities:
social, political, cultural, economic, race/ethnic, gender, age,
religion and disability values to unmask those that sustain an
oppressive status quo
Epistemology
Issues of power & Interactive link; knowledge is socially and
historically located; trusting relationship.
Methodology
Qualitative (dialogic)/ Quantitative / Mixed Methods;
Context
Mertens MESI March 2015
2/19/2015
3
Phase 1
Phase 2
Qual:
Qual:
Identify
Interview
stakeholder
s, focus
s; build
groups,
relationship
town
s; analyze
meetings
documents
Quant:
Identify
Quant:
available
Establish
demographic
baseline for
and
pollutants
Mertens MESI March 2015 and health
environmenta
l data
Phase 3
Qual:
Develop
interventi
ons based
on
stakehold
er input
Quant/Qual:
Pilot test
intervention
s & data
collection
Phase 4
Phase 5
Quant: process
assessment of
implementatio
n; pretests
Quant: Posttest
measures
Qual:
observe and
interview
stakeholder
groups
during
intervention
Qual:
Interviews,
observations,
policy
2/19/2015
4
planning
 Explicit goals?
 Implicit goals?
 Contradiction between explicit goals and behaviors?
 Changes in stance over time; changes in personnel
over time
 Explicit: Deaf people in the courts
 Implicit: I don’t really need to deal with these
multicultural issues because we are all white and well
educated/ of British extraction
Mertens MESI March 2015
2/19/2015
5
 “ism” present: depicting intended beneficiary as
“less than” member of powerful majority
 assuming problem is knowledge-based – not
social/cultural/oppression
 Problem: Self esteem; they just don’t care; have
babies to get welfare $$
 Problem: Ignorance (people with DA voting in
Pakistan; HIVAIDS in Botswana)
Mertens MESI March 2015
2/19/2015
6
 Tell them what they need to know, e.g., you have a right to vote and this is how you
do it; you can say no or use a condom
 Acknowledge the complexity of the solution that is needed as compared to the
services that are offered; develop partnerships to address complexity
 Use prior research and evaluation and applicable theories (e.g., developmental
psychology or change theory); best practices
 Consult with community about potential solutions: Address the meaning of the
experience in their lives (HIVAIDS), connect solution to experienced needs
(dancing, healthy cooking)
Mertens MESI March 2015
2/19/2015
7
 Who to involve? How to involve?
 Involvement throughout entire cycle of study:
questions, methods, data collection, use
 Notice: who is not here? Deaf people? Only people
who I am comfortable with?
Mertens MESI March 2015
2/19/2015
8
 Use low tech radio shows to solicit community members to
make contact via mobile phones; do follow up interviews
 Use smartphones for short surveys
 Web based recruitment; data collection and dissemination
 GPS and mapping
 An uncomfortable topic: sex with children; drug dealing;
gang activity; sex with same sex partner
Mertens MESI March 2015
2/19/2015
9
 Appeal to shared desire to provide effective services
addressing the needs of marginalized populations: process
and impact evaluation
 Argue for the primacy of the needs of the community and
disadvantaged clients
 Find a champion
 Build a coalition
 They don’t want this type of service; they won’t come….
Mertens MESI March 2015
2/19/2015
10
 Finding higher ground
 What are you going to do if you have negative
findings?
Mertens MESI March 2015
2/19/2015
11
 Most Significant Change: Davies and Dart (2005)
 ‘During the last month, in your opinion, what was the most
significant change that took place for participants in the
program?’ p. 10
 Other PAR strategies (Robert Chambers)
 Know the literacy level; comfort in speaking in groups
 Inuit culture: Children would not speak to the evaluator.
Mertens MESI March 2015
2/19/2015
12
 How to keep people safe?
 Sex with same sex partner is not accepted in
culture, is against the law, could be punished by
death
Mertens MESI March 2015
2/19/2015
13
 Wealth and status, gender, ethnicity, age group, sexual orientation, disability,
deafness, refugee, immigrant, religion
 Homogeneous groups; individual interviews
 Supportive accommodations
 Respect and meaningful involvement
Mertens MESI March 2015
2/19/2015
14
 Present actionable results; identify clear options for
possible results and associated actions
 Identify strategies to overcome barriers and support
change
 Share results in clear and understandable manners to full
range of stakeholders
 Share results with those who can advocate for change
(Cook, 2014)
 South Asia policy makers and program developers
Mertens MESI March 2015
2/19/2015
15
 Form coalitions
 Press releases
 Transparency
 Present to multiple audiences in multiple formats
Mertens MESI March 2015
2/19/2015
16
Advocate for good evaluation (standards);
advocate for justice
Evaluators have to be objective; they can’t be
advocates.
Mertens MESI March 2015
2/19/2015
17
 Mertens, D. M. & Wilson, A. (2012).
Program Evaluation Theory and
Practice: A Comprehensive Guide. NY:
Guilford.
 Mertens, D. M. (2015). Research and
evaluation in education and psychology:
Integrating diversity with qual, quant
and mixed methods. 4th ed. Thousand
Oaks, CA: Sage.
 Mertens, D. M. (2009). Transformative
research & evaluation. NY: Guilford.
 Mertens, D. M. & Ginsberg, P.
(2009).(Eds.) Handbook of Social
Research Ethics. Thousand Oaks, CA:
Sage.
 American Evaluation Association
(2011). Public Statement on Cultural
Competence in Evaluation. AEA.
Mertens MESI March 2015
2/19/2015
18
Donna M. Mertens, PhD
Independent Consultant
Donna.Mertens@Gallaudet.edu
Mertens MESI March 2015
2/19/2015
19