eSET
Stefani Dawn
Assistant Director of Assessment
Office of Academic Program, Assessment and
Accreditation
Decisions for Faculty Senate
HIGHEST PRIORITY DECISIONS
• Minimum class size for evaluations
• Combined or uncombined reporting
eSET-RELATED ITEMS and PROCESSES
• Demographic data collection
• Courses to include or exclude in eSET
• Financial, resource and technological limitations of eSET to be
configured for other evaluation approaches (formative evaluations
or department-centric evaluations)
Minimum Class Size for
Evaluations
OAR 576-003-0070
If Oregon State University solicits or accepts student survey evaluations of the classroom or
laboratory performance of a faculty member, the survey evaluations shall be conducted
anonymously…
History
•
Requests were made to have separate reports for slash courses. That was implemented
last term and quickly put on hold when it was discovered that some of the cross listed
sections had only a few students in it, even if all of the sections combined was a large
number.
•
Took the issue to the Faculty Senate EC to make some interim decisions to ensure that
the OARs are upheld.
Minimum Class Size for
Evaluations
Interim Decision
• Courses with fewer than 6 students will not be evaluated.
•
•
OSU guidelines
University of Oregon does not conduct evaluations in courses with 5 or fewer
students.
• Question – What should the number be to protect student
anonymity?
Factors to Consider
• Student input
Reporting: Combined or
Uncombined
•
•
There are advantages and disadvantages to combining reports.
Combining reports preserves data
•
•
Some faculty and departments prefer the combined data others do not
•
•
•
Example: 2 section slash course – one has 5 students, one has 4 students – combining
would allow the faculty members to receive the evaluations.
Differences in P & T forms/data per department
Graduate versus undergraduate
Having separate reports and combining data after the fact is not straight
forward
•
•
Median calculation is not a standard approach and cannot be done easily in an Excel
spreadsheet by the department or individual instructor
The Office of APAA and Enterprise Computing does not have the resources to
combine reports that have been separated.
The Median Calculation
•
Selected by the Faculty Senate in 2004
•
Based on "Measurement and Evaluation in Psychology and Education", Fourth Edition, by Robert L. Thorndike
and Elizabeth P. Hagen.
•
This calculation assumes that multiple values within a range are distributed evenly over the range. Say you have
the following distribution:
Score
6
5
4
3
Freq
29
29
38
10
Cumulative Frequency
106
77
48
10
The total responses is 106, so half of that is 53.
53 falls between the cumulative frequency of 48 and 77 corresponding to the scores 4 and 5.
53-48 = 5 more cases in the “5” range are needed. Since there are 29 responses for “5”, we need 5/29 of
them or .20 (rounded from .17).
We look at the range 4.5 – 5.5 and add .2 to 4.5 to get 4.7 as the median.
Demographic Data
Previous demographic data collected:
Gender:
The reason you are enrolled in this course:
Grade you expect to receive in this course:
Class Status:
Is this course your major?
Percent of this class you attended:
Your overall grade point average:
•
Students are still potentially identifiable by:
•
Gender
•
class status (e.g. the only graduate student in an undergraduate course)
•
Grade: Audit
Demographic Data
Factors to Consider
• Do faculty use the data?
• Survey fatigue by the students
Everything is An “If-Then”
•
If you decide to combine reports:
then how will that impact departmental processes for P & T?
•
•
If you decide to have separate reports:
Then what data will be lost? (Which is more important the data or the separate reports?)
•
Saving data and accumulating it over time is not an option for two reasons
•
•
•
Then how would reports be recombined (e.g. a person has 7 different slash versions of a course)?
•
•
•
OAR 576-003-0070 - Survey instruments from which evaluation data are obtained shall be delivered to the
faculty member.
There are over 4000 courses and >1500 faculty evaluated per term. Tracking this types of data is not feasible
with the current FTE arrangements.
Could change the median calculation to a standard calculation
If you decide to keep the demographic data
•
•
•
•
Then how might it compromise anonymity?
Is it information really used and needed?
Are there potential issues in the way questions are asked and student response to those questions?
Then how might it impact survey fatigue?