Participant-Oriented
Evaluation
Prepared by:
Daniel Wagner
Jahmih Aglahmie
Kathleen Samulski
Joshua Rychlicki
Participant-Oriented Evaluation



Evaluation approach focusing on the
participants
Participants have interest in the results
Developed in response to the needs
and interests of those associated with
the evaluation
Characteristics of a ParticipantOriented Approach




The evaluation is dependent on
inductive reasoning
The evaluation utilizes data from
multiple sources
The evaluation does not follow a
standard plan
The evaluation records multiple pointof-views or perspectives
An Evolved Approach
Other evaluation approaches
have been adapted into
participant-oriented
approaches.
Three Theorists Involved in
Approach



Robert Stake
Egon Guba & Yvonne Lincoln
Michael Q. Patton
Robert Stake




First evaluation theorist to introduce Participant
Oriented Evaluation into the field of education.
Introduced the theory of Responsive Evaluation which
focuses on “re-directing data gathering and
interpretative efforts around emerging issues of
importance to program practitioners in the evaluation
setting” (Abma, 2005)
Created the Countenance Framework in 1967. The
model refers to the two faces of evaluation: description
& judgment.
Description includes the evaluator’s observation and list
of benchmarks for the activities being evaluated.
Judgment is the evaluator’s overall rating of merit.
Guba & Lincoln



Built on Stake’s Responsive Evaluation theory and came up with
the method of fourth generation evaluation. This method was used
when listing the generations of evaluation.
Generations include:
- 1st generation: evaluators measure participants
- 2nd generation: evaluators describe participant
- 3rd generation: evaluators judge participants
- 4th generation: evaluators respond to participants.
The evaluation includes 12 steps.
1.
Contacting
7.
Prioritizing Unresolved Items
2.
Organizing
8.
Collecting Information
3.
Identifying Stakeholders
9.
Preparing for Negotiation
4.
Developing within-group Joint Constructions
10. Carrying Out Negotiation
5.
Enlarging joint stakeholder constrictions
11. Reporting
6.
Sorting out Resolved Claims, Concerns, and
Issues
12. Recycling
Michael Patton



Identified more than 100 evaluations
Known for his words: “Research is aimed at
truth. Evaluation is aimed at action”
He focuses on ‘identifying and working with
key participants in organizations who can
develop a vision for the value of gathering
information and using it to improve the
functions of the organization in ways
responsive to each situation.’ (Williams, 2000)
Key Considerations







(Fitzpatrick. Sanders, and Worthen, 2004), report that most
proponents of participant-oriented evaluation view the participants in
the object of evaluation as key to the evaluation.
As opposed to evaluating in a top-down manner, participant-oriented
evaluation is conducted with a bottom-up approach
Qualitative methods are used much more than quantitative methods.
Jacobs (1985) describes that qualitative methods, “seek to describe
not the frequency but the behaviors, beliefs, and feelings of the
respondents in a setting” (p. 5).
Jacobs continues to describe qualitative data as “direct quotations,
observations, personal impressions, analyses of records and
documents, summaries of unobtrusive measures, case studies, and
the like” (pp. 5-6).
Jacobs maintains the naturalistic design process in participantoriented evaluation is emergent and the setting is changeable and
natural.
As Fitzpatrick, et al. report, “the evaluator is in the role of the
learner, and the stakeholders serve as teachers” (p. 149).
Advantages of This Approach

Gain new insights and usable theories
–

Flexibility
–
–
–

Addresses user training as a feature of the assessment
Consideration to related variables
Numerous data collection techniques
Empowers stakeholders
–

Educational, social or corporate programs
Reaches even those who are at times left out of the
evaluation process
Add a political constituent
–
Cultivates activism
(Fitzpatrick, Sanders, and Worthen, 2004)
Disadvantages of This Approach

Subjective
–




Relies heavily on human observation and personal
perspective
Expensive
Labor-intensive
Time-consuming and extensive
Evaluator loses objectivity
–
may become caught up in advocacy trends
(Fitzpatrick, Sanders, and Worthen, 2004)
Implementation of ParticipantOriented Evaluation
When to use
 When cost is not an issue
 When time allows
 When responding to stakeholders
 When evaluator can take on the role of the learner
When not to use
 When detailed quantitative data is essential
 When there is “controlled, laboratory or contrived
setting” (Jacobs, 1985).
Scenarios for Use of ParticipantOriented Evaluation

Social reform
–

Education
–
–
–

Evaluating new and existing training programs
Government
–

Evaluating new and existing instructional programs on all
levels
Evaluating instructional materials
Evaluating professional development
Corporations
–

Evaluating programs developed to improve the
environment
Evaluating programs
Whenever qualitative formative evaluation is conducted
References
Abma, T.A.(2005). Responsive Evaluation. Evaluation and Program
Planning: Online version. Retrieved September 28, 2007 from World
Wide Web
http://64.233.167.104/search?q=cache:DyCD4R_ToE0J:mailer.fsu.e
du/~sullivan/SEA_Newsletter/Responsive_Evaluation.pdf+what+is+r
esponsive+evaluation&hl=en&ct=clnk&cd=6&gl=us
Fitzpatrick, J., Sanders, J., & Worthen, B. (2004). Program Evaluation:
Alternative Approaches and Practical Guidelines, (3rd Ed.), Boston:
Allyn & Bacon.
Jacobs, R. L. (1985). Naturalistic Inquiry and Qualitative Methods:
Implications for Training and Development. Columbus, OH: Ohio
State University. (ERIC Document Reproduction Service No.
ED305518).
Williams, D. D. (2000). Evaluation of learning objects and instruction
using learning objects. In D. A. Wiley (Ed.), The Instructional Use of
Learning Objects: Online Version. Retrieved September 27, 2007,
from the World Wide Web:
http://reusability.org/read/chapters/williams.doc