Successful Online
Implementation
Depends on a Culture of Trust
Bethel University College of Arts and Sciences
Dr. Richard Sherry
Dean of Faculty Growth and Assessment
IDEA Use at Bethel, 2006-07
Type
Diagnostic, Paper
Sections Percentage
39
4.5%
Short form, Paper
117
13.5%
Diagnostic, Online
Short form, Online
272
437
31.4%
50.5%
Total
865
Shift to Online Evaluation
Increasing Online Participation
DiagP
ShortP
DiagO
ShortO
100%
90%
80%
70%
60%
50%
40%
30%
20%
10%
0%
fall 2005
spring2006
fall2006
spring 2007
Normal Use of IDEA
• Context of faculty evaluation
– History of use at Bethel
• General policies for use
– Known policies that allow flexibility
– General confidentiality
– Use in a formative context
– Evaluation and multiple voices
– Acknowledgment of value and consequences
Who speaks on teaching?
Annual Reappointment
• Faculty selfassessment
• Department chair
observation, review
• Course evaluations
• Additional
observations if
desired
•
•
•
•
•
•
Promotion/Tenure
Extended selfassessment
Department chair
Course evaluations
3 peers, including
teaching
6 alumni
Additional if desired
Faculty Development and IDEA
• New Faculty Orientation*
• Guidance to faculty on interpretation and
application
• Approaches by personnel committees to
interpretation* CRI
• Follow up with Chair, Faculty Development
Coordinator, or Dean
• Direction for specific sessions on
pedagogy, curricular discussions
Committees Reading IDEA
• Understand the scale
• Take the long view (trends, types of
classes)
• Understand the context (kind of course)
• Consider response rates and distribution
• Look for key indicators
• Consider the objectives
• Read the comments
FDI
Faculty Concerns about Ratings
•
•
•
•
•
•
•
Earlier skeptical research repeatedly cited
(google: misconceptions "student course evaluations")
IDEA’s approach and “comparison”
Student ratings of instruction and the
“amenity” culture 9
“One bad course”
Time invested
Ceiling effect
Exploring a new course and “grace”
NO OTHER PROFESSOR uses the so called
"university wide grading scale". A’s are generally
accepted to be a good grade, and C’s are not good.
No matter what you might think. This is a
completely ridiculous grading system, as it makes
the GPA go down in relation to other classes that
don’t use this same scale.. . . Everyone else I know
in other college writing classes is getting an A. I’m
not saying that everyone deserves an easy A, but I
know that I, and many other students work really
hard to improve paper after paper, and it just
doesn’t happen.
FCR
Pilot of IDEA Online
and Growing Acceptance
•
•
•
•
•
The context for IDEA Online: reasonably
robust technology
Recruiting pilot cases
Reasonable lead time
Massive follow-up
Successful reporting
Engagement with Faculty
on Concerns
•
•
•
Response rates
Ratings differences
“Ganging up” and “unsupervised”
evaluations
Continuing to respond to technology
and faculty
•
•
•
Improvements in our processes
Customizing group reports
Customizing central questions
–
–
–
How has this course helped you connect this area
of study to a Christian perspective?
How has this course contributed to your
development as a thinking Christian?
How do you feel this course is influencing your life
as a believer?
Anatomy of a Failure
Implementation at College of Adult and
Professional Studies/ Graduate School
• Neglect of the context
• Inattention to politics
• Failure to prepare students and faculty
and communicate well
Reminders
•
•
•
•
•
Communicate policies clearly
Be appropriately flexible about evaluation
Clarify “voice” and “weight” of ratings
Create formative context
Provide guidance on implementation and
interpretation
• Provide guidance on committee use
Reminders (2)
• Communicate to students on value of
ratings
• Manage response rates through
monitoring technology
• Prepare students for the forms
• Reassure supervisory staff