Institutional Course Evaluation
Solution
Faculty Senate Executive Committee
September 12, 2012
Carol VanZile-Tamsen, Ph.D.; Associate Director, Office of
University Accreditation and Assessment
Joseph Mantione, Assistant Vice Provost, Institutional
Analysis
Currently at UB:
• Five different course evaluation solutions are
used across the 12 units
• Additional systems used by academic support
units delivering instruction
Currently at UB:
• In some areas, preparation of evaluation forms
for course offerings is an administrative
nightmare
• Students enrolled in courses offered by
different units do not have a single place in
which they evaluate all of their courses
• In many areas, response rates are too low for
any meaningful conclusions to be drawn
Currently at UB:
Solutions
#
Schools
ConnectEdu
Paper
In-House
SmartEvals
e*Value
4
3
2
2
1
College of Arts and Sciences, Dental Medicine, Nursing, Public Health
Architecture and Planning, Education, Law
Engineering, Medicine
Management, Social Work
Pharmacy
ConnectEdu in the College of Arts and Sciences
UBCATS - “University at Buffalo Course and Teacher Survey” - A web-based software application
that allows students to anonymously evaluate courses and instructors online.
Data Upload Process
Upload of Course, Enrollment, Faculty, Student, TA, and Supplemental Instructor tables to the site
Fall 2011
71,658
17,095
2,350
1,212
396
Student Enrollment Records
Student Import Records
Course Import Records
Faculty / Department Records
Teaching Assistants Records
Departmental Access Process (Dept. Chair, Assistants)
Set up the survey – keep it open for a month – close it two days after the last day of final exams.
ConnectEdu in the College of Arts and Sciences
Communication Process
•
•
•
•
•
Place student announcement and survey link on myUB
Send Emails (17,078 initial emails) to Students to:
Inform them that the evaluation site is open
Invite them to log in and do the evaluation).
Send Emails (1,137 initial emails) to Faculty to:
Inform them that the survey is open
Ask them to monitor response rates (real time)
Remind their students to do the evaluation
Send follow-up weekly emails/reminders to students with incomplete surveys
(15,719, 14,072, 12,652, 11,733 emails in Fall’11)
Send final Email to Faculty at the close of the survey to let them know that they can log in and
access the evaluation results the day after web grading closes for the semester.
Reporting Process
Distribute our own custom Summary Reports to Dept. Chair and Director of Graduate Studies.
ConnectEdu in the College of Arts and Sciences
UBCATS Challenges
1
2.
3.
4.
5.
6.
7.
8.
9.
10.
Data upload- creation of new HUB querying tools
Undeliverable Emails – students not using/checking their UB email
Lack of Summary Reports
Departmental buy in – lack of ownership
Limited technical support
Confusion among students – not all UB courses are part of UBCATS
I can’t log in!
Reports are not accessible to students
Response rate
Survey fatigue
To some degree, the other 11 schools, Athletics, MFC and EOC all do the same thing
and incur the same PSR costs.
They also have the same challenges.
The Benefit of A Single Solution:
• Uniformity in Administration leading to greater
efficiency and more reliable results
• Greater transparency for students and one-stop
evaluations may encourage greater response rates
• Availability of Data and Reports for Assessment
Purposes:
– Faculty review and development
– Program Review
– Accreditation Reporting
Course Evaluation Items:
• For comparability of ratings across courses and
instructors, a core of common questions could
be useful:
– 3-5 questions
– These would be developed jointly by Faculty Senate
Executive Committee and Associate Deans Council
• Beyond that, we would promote the greatest
flexibility possible for units in selecting
questions
Vendors already on campus
Campuslabs
http://www.campuslabs.com/ (cost proposal submitted)
ConnectEdu
http://www.connectedu.com/postsecondary-solutions/collegeevaluation-suite/college-course-evaluation.html
(cost proposal submitted)
Digital Measures
http://www.digitalmeasures.com/course-evaluation-software/
E*Value
https://www.e-value.net/
SmartEvals
http://service.onlinecourseevaluations.com/
*Blackboard
http://www.blackboard.com/ (no course eval solution at present)
Conclusions
• Regardless of platform chosen, there are many
good reasons to have an institutional course
evaluation system
• We hope you will consider data use and
reporting reasons, as well as fiscal reasons, as
you consider this issue.