PARADIGM
-a model of something, or a very clear and
typical example of something
-an outstandingly clear or typical example
or archetype
PARADIGM
-A paradigm is "a world view, a way of
ordering and simplifying the perceptual
world's stunning complexity by making
certain fundamental assumptions about the
nature of the universe, of the individual,
and of the society.
PARADIGM
-made up of four sets of assumptions that
address people’s perspectives of what is
ethical, what is real, what is considered to
be valid knowledge and what is considered
to be appropriate evaluation methods
PHILOSOPHICAL ASSUMPTIONS
ASSUMPTION
GUIDE QUESTION
AS EXPERIENCED IN LIFE
1. Axiology
What is the
nature of
ethics?
Your societies,
cultures and
religion taught
you morals of
what is right or
wrong.
PHILOSOPHICAL ASSUMPTIONS
ASSUMPTION
GUIDE QUESTION
AS EXPERIENCED IN LIFE
2. Ontology
What is the
nature of
reality?
Are there realities
that differ,
depending on the
experiences and
conditions of the
people in a
specific context?
PHILOSOPHICAL ASSUMPTIONS
ASSUMPTION
GUIDE QUESTION
AS EXPERIENCED IN LIFE
3.Epistemology
What is the nature
of knowledge and
what is the
relationship
between the
knower and that
which would be
known?
How should the
evaluator,
objectively
stand apart from
the
stakeholders?
PHILOSOPHICAL ASSUMPTIONS
ASSUMPTION
GUIDE QUESTION
AS EXPERIENCED IN LIFE
4. Methodology
What are the
systematic
approaches to
gathering
information about
what would be
known?
Should you use
quantitative or
qualitative? Do
you need to
compare two
groups?
EVALUATION THEORY
-that aspect that reflects our thinking about
how and why we engage in evaluation
-provides guidance in determining the
purposes of evaluation
CRITERIA
1. Knowledge: What do we need to do to
produce credible knowledge?
2. Use: How can we use the knowledge
we gain from an evaluation?
3. Valuing: How do we construct our value
judgments?
CRITERIA
4. Practice: What do we evaluators
actually do in practice?
5. Social programming: What is the
nature of social programs and their
roles in solving societal problems?
MODELS AND APPROACHES
-models/approaches can be thought of
as set of rules, prescriptions, and
prohibitions and guiding frameworks
that specify what good or proper
evaluation is and how should it be done
(Alkin, 2013)
MAJOR PARADIGMS IN EVALUATION
PARADIGM
BRANCH
Postpositivist
Methods
Pragmatic
Use
Constructivist
Values
Transformative
Social Justice
DESCRIPTION
Focuses on quantitative designs and data: may use
mixed methods but quantitative method is more
common
Focuses on data that are found to be useful for
stakeholders: advocates for the use of mixed methods
Focuses on identifying multiple values and perspectives
through qualitative methods: may use mixed methods
but qualitative method dominates
Focuses on viewpoints of marginalized groups and
interrogating systemic power structures through mixed
methods to further social justice and human rights
POSTPOSITIVIST PARADIGM
AND
THE METHODS BRANCH
-holds to the belief in quantitative
approaches;
--keeps distance from the object of study
-strives to be objective by limiting the
contact or involvement with people in
the study
-conduct of good research is a
fundamental requirement for ethical
conduct
(good research-reflects intellectual
honesty, the suppression of bias and
careful collection of empirical studies
-axiological assumption is closely aligned
with the ethical principles
1. beneficence
2. respect
3. justice
Mark and Gamble, (2009) suggest that
the methodological choice of a
randomized experimental design is
ethically justified when the purpose of
the study is to establish cause-and-effect
relationship
EARLY THEORISTS AND THEORIES
Ralph Tyler
Ralph Tyler
-used the term “educational evaluation” back in
the 1930s, making him one
of the earliest scholars in this field
- approach consisted primarily of establishing
educational objectives and then determining
whether those objectives had been met
PETER ROSSI
Peter Rossi
-contributed his thinking to evaluation from the perspective of
evaluation research in the form of survey methods and social
science experiments
-began writing about the connection between evaluation and
social policy in the early 1970s
Hecontinued making contributions along this line through his
multiple books
Peter Rossi
-strength of his contribution is in the
explanation of how randomized control designs
and quasi- experimental designs could be used
in evaluation, and how
evaluation findings could be tied to national
policy in education and human services
Gary Henry
and
Melvin Mark
Gary Henry and Melvin Mark
-extended theories of evaluation in the
context of the use of causal modeling for
evaluation, with particular attention to ethical
issues
They also presented extensions of a theory of
evaluation that Pawson and Tilley (1997)
discussed, known as “emergent realist
evaluation”
(ERE) theory (Henry, Julnes, & Mark, 1998).
THEORY-BASED EVALUATION
-an approach that focuses on the
theories people have about
what it takes to have a successful
program
Huey-Tsyh Chen
-worked with Peter Rossi to develop the
concept of theory-based evaluation as the
logical extension of quantitative models
that permit identification of variables
contributing to the outcomes of a program.
Stewart Donaldson
-offers a change in labeling for evaluations that have
program theory at their core
defines “program- theory-driven evaluation
science” as “the systematic use of substantive
knowledge about the phenomena under investigation
and scientific methods to improve, to produce
knowledge and feedback about, and to determine the
merit, worth, and significance of evaluands such as
social, educational, health, community,
and organizational programs”
EVALUATION OF TRAINING
PROGRAMS
The “Kirkpatrick model” for evaluation of training
programs dominated human resource
development evaluations for many decades
It has four levels of evaluation:
participant reactions, learning, behavior, and results.
This model was extended to consider the financial
return on investment (ROI) of training by Phillips
(reaction evaluation)
-Brinkerhoff (2003) also developed an impact model
for training evaluations -“success case method”
includes both quantitative and qualitative data
-he recommends its use in contexts in which a full
experimental study is not feasible.
Theory to Practice: Methods Branch
1. Experimental and quasi- experimental
approaches
2. Theory-based evaluation
3. Evaluation of training programs
Quantitative Research
Random Assignment
Control Group
Experimental group
Independent variable
Dependent variable
Quasi- experimental methods-The two groups studied in
an experiment are nonequivalent and do not involve random assignment
to the experimental and control groups
Quantitative Research
Random Assignment
Control Group
Experimental group
Independent variable
Dependent variable
Quasi- experimental methods-The two groups studied in
an experiment are nonequivalent and do not involve random assignment
to the experimental and control groups