eSET Timing Close Study Results
Stefani Dawn, PhD
Assistant Director of Assessment
Background
• Requests have been made to modify the
closing date of eSET.
– close before finals
– close after finals but before grades are released
(with grades access blocked until evaluations are
completed)
– close at the end of grade week, with access to
grades blocked until evaluations are completed
Data Collected
•
•
•
•
Response rates for each of the three scenarios
Median scores for each of the three scenarios
Survey results
Logistical considerations
Response Rates for
3 Different Close Timings
Term
Answered
Declined
Total
Response
Rate
Incomplete
Fall 2011
67%
22%
12%
(n=101,042)
(n=32,872)
89%
66%
24%
(89,363)
(32,403)
Winter
2012
Spring
2012
58%
22%
(71,069)
(27,253)
Fall 2012
60%
13%
(88,702)
(19,449)
45%
8%
(n=62,930)
(n=10,933)
Winter
2013
90%
80%
73%
53%
(17,732)
11%
(14,515)
19%
(23,646)
27%
When Ended (Holds
Were Placed on
Viewing Grades)
End of grade release
week
End of grade release
week
after finals before
grades posted
(40,712)
after finals before
grades posted
47%
Before finals
(n=64,799)
Median Scores
Term
Median – Q1
“The course as a whole
was:”
Median – Q2
“The instructor’s contribution to a
course was:”
Fall 2011
4.7
4.9
Winter 2012
4.8
5.0
Spring 2012
4.9
5.1
Fall 2012
4.8
5.0
Winter 2013
4.8
5.0
Survey Results
Timing Preference by %
– Faculty/staff (n = 31), Students (n=1669)
45%
40%
35%
30%
25%
Faculty/Staff
20%
Students/Other
15%
10%
5%
0%
Before finals
After finals but before End of grade-release week
grade-release
No preference
Survey Results
Comments Summary
• There were 275 comments submitted: 254
were from students
• Students:
– They are too busy studying and completing papers
to complete evaluations before finals
– The final is an important part of the course and
want to experience the entire course before
evaluating it
Survey Results
Comments Summary
• Students:
– In many cases, students had either not yet
received feedback from faculty on major
assignments, or instruction was still occurring
before finals
– They want to be able to provide thoughtful
feedback, which requires time.
Survey Results
Comments Summary
• Students:
– They recognize that faculty are concerned about
potential retaliation with evaluations after grades
are posted, and suggested closing before grades
were released or blocking the ability of students
to see their grades before evaluations are
complete.
• Likewise, students are very concerned about allowing
faculty to see evaluation reports prior to grades being
released.
Survey Results
Comments Summary
• Faculty:
– Concern about students seeing grades prior to
evaluations and retaliatory evaluations as a result,
or concerns about scores being lower after
students have completed a tough final.
– Concern about a low response rate.
Logistical Considerations
• When closing before finals
– Instructor added questions period is reduced.
– Reports will not be released until after grades are
posted.
Faculty Senate
Executive Committee
• Recommendation
– Close after finals but before grades are officially
posted on the web with grade access blocked.
• NOTE: If you post grades on Blackboard and do not
want students to view it then the instructor must hide
those.
• There are instructions on the APAA eSET website on
how to do this.