HEQCO Learning Outcomes Assessment
Consortium – University of Toronto
Overview
A team of researchers at the University of Toronto seeks to develop four-level analytic rubrics and
training materials to assess competencies in communication, teamwork, investigation, design and
problem analysis. Analytic rubric criteria refer to indicators, which are specific measurable learning
outcomes that indicate competency in a global outcome area. These rubrics must be flexible enough to
be implemented across a wide range of courses and institutions. The team is led by Susan McCahan,
Vice-Dean, Undergraduate and Professor of Mechanical Engineering in the Faculty of Applied Science
and Engineering at the University of Toronto.
Methodology and Timeline
The project contains three phases:
•
Phase 1| Literature Review and Environmental Scan (March 2013-January 2014)
This phase of the project lays the foundation for the development of the assessment instrument. Other
institutions have done similar work designing and developing rubrics, but there is little to no consensus
on how like terms are used. So, before preparing the assessment tool, the team combed the literature
and existing work to create a comprehensive list of learning outcomes and indicators under the five
competencies they seek to assess. These results guided the development of the four-level analytic
rubrics.
•
Phase 2 | Rubric Development and Modification (February 2014-August 2014)
During this phase, a set of four-level analyst rubrics are being developed based on the indicators
generated from Phase 1. These will undergo extensive review by an educational developer, instructors
and departmental administrators.
•
Phase 3 | Rubric Testing in Engineering Courses (September 2014-March 2015)
The rubrics will be piloted throughout the nine engineering programs at the University of Toronto. The
team will work with instructors to integrate the rubrics into existing projects, assignments and tests as
to not create any additional work for the student. Over the course of the year, the rubrics will be tested
for inter-rater reliability and validity and will be modified accordingly. Additionally, student focus groups
will be conducted to assess whether the students understand the feedback provided by the rubrics in
the way intended by the instructor.
•
Phase 4| Expansion of Rubric Testing (April 2015-May 2016)
In order to assess the extent to which these rubrics are scalable across the institution, they will be
deployed at partner institutions and other faculties within the University of Toronto.
Successes and Challenges
From the outset of the project, the main challenge has been identifying a common learning outcomes
nomenclature. Establishing definitions will be important, not only for the broader uptake of the rubrics
across faculties within the university, but also for their use across institutions.
Contact Information – For more specific information about this project or for any questions, please
contact Fiona Deller, Executive Director, Policy and Partnerships at HEQCO, fdeller@heqco.ca or Susan
McCahan, Vice-Dean, Undergraduate at the University of Toronto, mccahan@mie.utoronto.ca.