Institutional Evaluation Framework – Open UToronto MOOCs DRAFT

Phase 1 – Needs Assessment and Planning
o Work with lead MOOC instructors to identify common Scholarship of Teaching and
Learning SoTL research questions across MOOCs and related blended in-class
initiatives.
o Identify additional Institutional Impact research questions for other aspects such as
PR impact, faculty development, capacity development, resourcing needs.

Phase 2 – Institutional Research program
o Coordination of efforts across all MOOCs/blended/inverted projects would be
undertaken to ensure activities and data collection processes are coherent.
o Anticipate individual faculty research will simultaneously move ahead in discipline
specific SoTL projects.

Phase 3 – Coordinating Reporting (Institutional and Individual)
o Coordinated efforts to ensure that we share/publish/make visible these institutional
and individual research outputs (research impact external to the university, but
also internal to inform planning.)
Possible Research Questions:
A. Teaching and Learning
MOOCs
o Evaluate effectiveness and scalability of pedagogical approaches and learning activities
delivered via open course platforms. Example dimensions:
 range of learning activities across discipline areas
 effectiveness of assessment methods
 user data analytics
 what are the affordances (opportunities and challenges) of online open courses as
they emerge from the participants perspectives
 Other metrics/dimensions TBD
Blended/Inverted Classroom Activities
o Evaluate potential to leverage integration of open educational resources/platforms/tools in
degree program courses. Example dimensions:
 Effectiveness of various formats/design models
 impact on learning outcomes
 Student engagement/course evaluation data
 Institutional resourcing/support needs
 Other metrics/dimensions TBD
B. Institutional Impact
Communication
o Assess potential to enhance the University of Toronto’s reputation as a leading institution
for teaching, learning and research. Example dimensions
 Strat Communications analytics
 Research outputs
 Internal impact – increase in attention/participation in activities related to online
learning innovation in this arena.
Capacity Building
o Challenges and Opportunities for Dept. MOOC/course development teams
 Faculty perception
 Faculty development needs/uptake
 Production capacity expansion/ed tech professionals
 Linkage to online learning strategy (hybrid/online degree course development)
 Resourcing implications