eSET Timing Close Study Results Stefani Dawn, PhD Assistant Director of Assessment Background • Requests have been made to modify the closing date of eSET. – close before finals – close after finals but before grades are released (with grades access blocked until evaluations are completed) – close at the end of grade week, with access to grades blocked until evaluations are completed Data Collected • • • • Response rates for each of the three scenarios Median scores for each of the three scenarios Survey results Logistical considerations Response Rates for 3 Different Close Timings Term Answered Declined Total Response Rate Incomplete Fall 2011 67% 22% 12% (n=101,042) (n=32,872) 89% 66% 24% (89,363) (32,403) Winter 2012 Spring 2012 58% 22% (71,069) (27,253) Fall 2012 60% 13% (88,702) (19,449) 45% 8% (n=62,930) (n=10,933) Winter 2013 90% 80% 73% 53% (17,732) 11% (14,515) 19% (23,646) 27% When Ended (Holds Were Placed on Viewing Grades) End of grade release week End of grade release week after finals before grades posted (40,712) after finals before grades posted 47% Before finals (n=64,799) Median Scores Term Median – Q1 “The course as a whole was:” Median – Q2 “The instructor’s contribution to a course was:” Fall 2011 4.7 4.9 Winter 2012 4.8 5.0 Spring 2012 4.9 5.1 Fall 2012 4.8 5.0 Winter 2013 4.8 5.0 Survey Results Timing Preference by % – Faculty/staff (n = 31), Students (n=1669) 45% 40% 35% 30% 25% Faculty/Staff 20% Students/Other 15% 10% 5% 0% Before finals After finals but before End of grade-release week grade-release No preference Survey Results Comments Summary • There were 275 comments submitted: 254 were from students • Students: – They are too busy studying and completing papers to complete evaluations before finals – The final is an important part of the course and want to experience the entire course before evaluating it Survey Results Comments Summary • Students: – In many cases, students had either not yet received feedback from faculty on major assignments, or instruction was still occurring before finals – They want to be able to provide thoughtful feedback, which requires time. Survey Results Comments Summary • Students: – They recognize that faculty are concerned about potential retaliation with evaluations after grades are posted, and suggested closing before grades were released or blocking the ability of students to see their grades before evaluations are complete. • Likewise, students are very concerned about allowing faculty to see evaluation reports prior to grades being released. Survey Results Comments Summary • Faculty: – Concern about students seeing grades prior to evaluations and retaliatory evaluations as a result, or concerns about scores being lower after students have completed a tough final. – Concern about a low response rate. Logistical Considerations • When closing before finals – Instructor added questions period is reduced. – Reports will not be released until after grades are posted. Faculty Senate Executive Committee • Recommendation – Close after finals but before grades are officially posted on the web with grade access blocked. • NOTE: If you post grades on Blackboard and do not want students to view it then the instructor must hide those. • There are instructions on the APAA eSET website on how to do this.