Course Evaluations at BYU-Idaho
© 2008 Brigham Young University–Idaho
1
Primary Objectives of Course Evaluation
1.
Assess course quality and teaching
performance.
2.
Assist in teacher improvement.
3.
Give students an opportunity to
provide input
4.
Satisfy accreditation requirements
© 2008 Brigham Young University–Idaho
2
The Numbers
Classes Evaluated
Instructors Evaluated
Evaluations Administered
© 2008 Brigham Young University–Idaho
F09
W10
S10
1,514
1,406
1,264
494
509
485
44,794 43,035
39,996
3
Completion Rates
© 2008 Brigham Young University–Idaho
4
Completion Rates
College
F09
W10
S10
Agricultural & Life Sciences
65%
64%
62%
Business & Communication
69%
69%
67%
Education & Human Development
72%
68%
69%
Language & Letters
64%
66%
66%
Performing & Visual Arts
53%
53%
52%
Physical Sci & Engineeering
76%
78%
76%
Grand Total
66%
68%
68%
© 2008 Brigham Young University–Idaho
5
The Overall Instructor Rating
© 2008 Brigham Young University–Idaho
6
How Does the Administration Use The Data?
•
We aggregate and track . . .
•
•
•
•
•
•
•
Overall instructor rating
Overall course rating
Perceived learning
Hours of preparation
Satisfaction
Learning model scales
We use the overall instructor rating to . . .
•
•
•
•
Support CFS decisions
Compare classifications (online, adjunct, veteran, 1-year)
Watch trend
Red flag instructors at or below 10%tile
© 2008 Brigham Young University–Idaho
7
What the Research Shows – Non-factors
Time of Day
Instructor Traits
•
Student traits
Age
• Academic aptitude
• GPA
• Class level
• Personality
•
© 2008 Brigham Young University–Idaho
•
Age
Years of teaching
experience
8
What the Research Shows - Factors
Class Size
Smaller classes tend
to receive higher
ratings.
Reason for Taking
Course
Elective courses
receive higher ratings
than required courses.
Expected Grade
Positive but low.
© 2008 Brigham Young University–Idaho
Discipline
The highest ratings go
in the following order
to:
1.
2.
3.
4.
Arts & Humanities
Biological & Social
Sciences
Business & Computer
Science
Math, Engineering, &
Physical Sciences
9
What the Research Shows - Factors
Difficulty Level
Courses that are more
difficult or have greater
workloads received
higher ratings.
Motivation
Prior interest in subject
matter or class leads
to higher ratings.
Student Major
Course Level
Upper division receive
higher ratings than
lower division.
© 2008 Brigham Young University–Idaho
Majors are a bit more
positive.
10
What the Research Shows - Factors
Instructor Gender
Same-gender
instructors receive
slightly higher ratings.
Instructor
Personality
Knowledgeable, warm,
outgoing, and
enthusiastic teachers
receive higher ratings.
© 2008 Brigham Young University–Idaho
Scholarship
Teachers with more
publications receive
slightly higher ratings.
Timing
Administration during
final exams are lower.
PR
Framing re promotion
& tenure gets slightly
higher ratings.
11
DO THESE FINDINGS HOLD
FOR BYU-IDAHO?
© 2008 Brigham Young University–Idaho
12
Findings From 2009 – 2010 Data
© 2008 Brigham Young University–Idaho
13
Findings From 2009 – 2010 Data
© 2008 Brigham Young University–Idaho
14
Findings From 2009 – 2010 Data
© 2008 Brigham Young University–Idaho
15
Findings From 2009 – 2010 Data
© 2008 Brigham Young University–Idaho
16
Findings From 2009 – 2010 Data
Ag & LS
PS & Eng
© 2008 Brigham Young University–Idaho
Bus & Comm
Ed & HD
17
Findings From 2009 – 2010 Data
Gender of Student
Gender of Instructor
Female
Male
Female
5.85
5.69
Male
5.82
5.89
Overall
5.83
5.86
© 2008 Brigham Young University–Idaho
18
Findings From 2009 – 2010 Data
© 2008 Brigham Young University–Idaho
19
Findings From 2009 – 2010 Data
© 2008 Brigham Young University–Idaho
20
Dreams for the Future
•
•
Supplement with direct measures of learning
Adjust the composite measures for
•
•
•
•
•
•
•
•
Discipline
Gender
Reason for taking class
Grade Expected
Rework the Learning Model scores
Better reports
Shorten the instrument
Develop some information for students
© 2008 Brigham Young University–Idaho
21
Discussion Items
•
•
•
•
•
•
•
•
Are we asking the right questions?
Are we asking too many questions?
Should we evaluate every course every semester?
Is the timing of the evaluation optimal?
How do you use course evaluation data (numbers
and comments)?
Is the information valuable?
Are the reports adequate?
Should some of the course evaluation data be
made available to students to counter biased
information on the web?
© 2008 Brigham Young University–Idaho
22
© 2008 Brigham Young University–Idaho
© 2008 Brigham Young University–Idaho
Raw vs Weighted Averages
• 85% see little or no difference (-.1, 0, .1) = 85%
• 76 get a better rating; 21 get a worse rating
• 2 get a significantly worse rating
© 2008 Brigham Young University–Idaho
24