PROGRAM
QUALITY
ASSURANCE
PROCESS
AUDIT
18‐MONTH
FOLLOW‐UP
REPORT
DATE: 24/06/2013
Table of Contents
Introduction: ................................................................................................................................................. 2
Recommendations: ....................................................................................................................................... 3
Recommendation #1:................................................................................................................................ 3
Recommendation #2:................................................................................................................................ 3
Recommendation #3:................................................................................................................................ 3
Recommendation #4:................................................................................................................................ 3
Affirmations: ................................................................................................................................................. 4
Affirmation #1: .......................................................................................................................................... 4
Affirmation #2: .......................................................................................................................................... 4
Affirmation #3: .......................................................................................................................................... 4
Affirmation #4: .......................................................................................................................................... 4
Summary: ...................................................................................................................................................... 5
Table 1: Follow-Up on Recommendations: ................................................................................................... 6
PQAPA: 18 Month Follow-up Report
Table 2: Follow-Up on Affirmations: ........................................................................................................... 11
1
Introduction:
Seneca College underwent its second PQAPA audit in June 2011 with the final results of our
audit report confirmed in December 2011. The panel visit provided a valuable opportunity for
sharing of information and ideas. We are encouraged to have received such a positive,
independent assessment and to have been given constructive advice as we continue to build
upon the quality framework in place at Seneca. Substantial progress has been made in all areas
that led to the Audit recommendations. In response to the audit, and to address the
importance of quality processes, Seneca has formed a Program Quality Reference Group
(PQRG). Composed of representatives from the senior academic leadership (Associate Vice
Presidents, Deans, Chairs, and Directors) as well as faculty leads from the Centre of Academic
Excellence, the PQRG has informed the review and update of program quality assurance
policies and the development of new resources and tools relative to program quality. The
college has implemented numerous enhancements which ameliorate areas of our program
development process, faculty development, and summative and formative review processes.
The audit process has resulted in a heightened awareness of quality assurance processes at the
institutional, departmental and individual levels. As noted by the panel, the recent launch of
the Centre for Academic Excellence and the refinement of our quality assurance mechanisms
have been invaluable in our quest for continuous quality improvement. Our efforts will be
further aided by the hiring of a new Director of Program Quality, which is presently underway.
PQAPA: 18 Month Follow-up Report
We were pleased to receive 10 commendations from the panel for our work. In addition to the
commendations, the panel report outlined 4 recommendations and 4 affirmations. Since the
time of the audit, we have undertaken several measures to respond to the recommendations
and have continued our work to address those areas we identified to the panel through the
affirmation process. Our plan and progress against the recommendations and affirmations are
provided in Tables 1 and 2 of this report.
2
Recommendations:
The following lists the recommendations made by the Audit Panel at the time of the audit.
Details on the actions taken to address these recommendations are presented in Table 1 of this
report.
Recommendation #1:
The College should consider streamlining the Tier 3 process somewhat to make it less onerous
and less resource-intensive and to enable a shorter cycle of review completion. Perhaps setting
a goal of a formal review for every program every 5 years would better ensure currency and
relevancy of Seneca’s curricula.
Recommendation #2:
Though PLAR is advertised in multiple ways and means, the College might consider placing a
statement on every subject outline as to PLAR applicability.
Recommendation #4:
The College should consider setting an aggressive schedule for completing all remaining Tier 3
reviews and, regardless of environmental circumstances, this schedule should be respected and
maintained.
PQAPA: 18 Month Follow-up Report
Recommendation #3:
The College should ensure that all programs are mapped to the program learning outcomes and
that the annual review of each program should ensure that this mapping is regularly updated.
3
Affirmations:
The Audit Panel affirmed the following initiatives that appeared in Seneca’s Self-Study
document. Details on our activities and outcomes are presented in Table 2 of this report.
Affirmation #1:
The College’s commitment of intent to ensuring faculty and students’ awareness of program
learning outcomes and the more detailed mapping of final and major assessments by faculty is
encouraging.
Affirmation #2:
The College’s recent requirement to add Essential Employability Skills learning outcomes to all
subject outlines.
Affirmation #3:
The College’s recognition that it must either revise the Faculty Performance Review Policy or
find ways to help the Chairs adhere to this requirement.
PQAPA: 18 Month Follow-up Report
Affirmation #4:
The College’s recognition that it needs to provide professional development workshops on
writing learning outcomes appropriate to the credential and orientation sessions early in the
cycle to assist in the on time completion of the Report.
4
Summary:
As noted in the Panel’s report, there is renewed energy and enthusiasm with respect to quality
assurance initiatives at Seneca. We are confident that we will be able to sustain this
momentum in the coming years. We will continue to review and refine our policies, processes
and practices, and look forward to demonstrating our accomplishments to the panel at
subsequent PQAPA audits.
Seneca values excellence, innovation, community, and diversity. Our Strategic Plan (2012-17)
underlines our commitment to quality and improvement:


“We never compromise on quality, standards and results.”
“We are always improving, innovating and looking ahead.”
Seneca is committed to offering the highest quality programs and learning experiences to its
diverse student population. Our new emphasis on cross disciplinary, experiential and
networked learning, along with the Seneca Core Literacies, distinguishes our programs, the
Seneca experience and our graduates.
The provincial quality assurance process audit provided an opportunity for the college to reflect
on its processes, identify areas for growth, and to construct a plan for improvement, thus
keeping quality assurance at the top of our priorities.
For any additional information relating to this report, please contact Laurel Schollen, Associate
Vice
President
Academic
–
Educational
Excellence
at
Seneca
College:
Laurel.Schollen@senecacollege.ca 416.491.5050, Extension 22430.
PQAPA: 18 Month Follow-up Report
The tables appended to this report describe the follow up actions and evidence in response to
each affirmation and recommendation we received. The actual evidence samples are available
for viewing and validation upon request.
5
Table 1: Follow-Up on Recommendations:
TABLE 1: FOLLOW-UP ON RECOMMENDATIONS
Recommendation 1.
The College should consider streamlining the Tier 3 process somewhat to make it less onerous and less resource-intensive and to enable
a shorter cycle of review completion. Perhaps setting a goal of a formal review for every program every 5 years would better ensure
currency and relevancy of Seneca’s curricula.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
At the time of the audit, Seneca was working with the model of Quality Assurance approved by the Board of Governors in 2004. The process evolved, matured and
improved over the years. We concur with the panel that review and revision of the policy were needed to ensure meaningful and substantive reviews were completed
with appropriate resources allocated within allocated time frames.
WHO IS THE LEAD?
Associate Vice President
Academic, Educational
Excellence
WHAT IS THE
DEADLINE?
June, 2012
WHAT IS THE CURRENT STATUS?
Completed
Centre for Academic
Excellence
Provide more central resources to ensure high quality reviews
are completed within an appropriate timeframe.
June, 2012
Completed
A new Program Quality Reference Group has been
convened to provide input on our policies and
practices relative to program quality and to provide
recommendations to the appropriate college
bodies. The group is composed of a representative
group of Associate VPs, Deans, Directors and
Chairs,
PQAPA: 18 Month Follow-up Report
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Approve new quality assurance and program review policies.
New polices will mandate summative program reviews be
completed every 7 years (or more frequently as required by
regulatory bodies) and formative reviews biennially.
6
Ongoing
Streamline process and report templates; pre-populate report
shells to ensure high quality consistent reporting.
Ongoing
Orientate and schedule regular meetings for team leads to share
experiences and learn from each other.
PQAPA: 18 Month Follow-up Report
Increase efforts to complete all in-progress summative reviews
and create an aggressive schedule to ensure completion of all
remaining summative reviews within a 7 year cycle.
and representatives from the CAE. Past agenda
items have included: review of program quality
measures at other institutions, strategies to ensure
review processes are meaningful and lead to
continuous improvement, and identifying gaps and
redundancies in our formative and summative
processes. The committee has met 6 times to date
and will continue to meet 3-4 times per year.
- 7 year summative program review schedule
has been established. The schedule is
reviewed/adjusted annually by the Deans’
group.
- Outstanding summative reviews 97%
completed
- 100% (13/13) summative reviews completed
within 10 months in 2012
- 23 summative reviews on schedule for
completion by December 2013
- 30 formative reviews scheduled for 2013
- 30 summative reviews scheduled for 2014
Completed
- Program review orientation session developed
for all review team leads
- Institutional data packages are prepared well
in advance of review kick off to allow CAE to
pre-populate the report shells
- CAE provides training for program review leads
on the process, timelines, deliverables
- Draft reports are reviewed by CAE staff and the
AVPA prior to presentation to the VPA
7
Recommendation 2.
Though PLAR is advertised in multiple ways and means, the College might consider placing a statement on every subject outline as to
PLAR applicability.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
The College will undertake a review of the policies and practices relative to PLAR applicability and publication.
WHO IS THE LEAD?
Vice President
Academic
WHAT IS THE
DEADLINE?
December 2012
WHAT IS THE CURRENT STATUS?
Completed
In order to improve our communication of the Seneca
PLAR policy to our students, the College has created a
new online, automated system whereby students are
made aware of the PLAR policy and process, along
with other Seneca academic policies. Upon account
activation, all Seneca students are prompted through
an automated process which provides information
about Seneca policies and procedures (including
PLAR). Students are required to acknowledge the
information and are sent a confirmation e-mail that
includes a copy of links to the Seneca policies. The
new process was implemented college-wide in Fall
2012.
The College also ensured that information regarding
PLAR policies and procedures is included in full-time
and part-time calendars (both print and online), as
well as in Seneca's Academic Policy, Section 4.4.
http://www.senecacollege.ca/academicpolicy/acpol04.html
PQAPA: 18 Month Follow-up Report
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING
THE RECOMMENDATION?
Review Seneca's PLAR process and recommendation. Explore
opportunities to improve communication of Seneca policies
with students.
8
Recommendation 3.
The College should ensure that all programs are mapped to the program learning outcomes and that the annual review of each program
should ensure that this mapping is regularly updated.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
Review the various stages of new program development and both formative and summative review processes to determine the best way to incorporate mapping of
program learning outcomes into the process.
Review mapping tool and practices conducted in summative
review.
Explore strategies to incorporate mapping into formative review
process.
WHO IS THE LEAD?
Associate Vice President
Academic, Educational
Excellence
Centre for Academic
Excellence
WHAT IS THE
DEADLINE?
March 2013
WHAT IS THE CURRENT STATUS?
Completed
A new program development process has been
developed and implemented. Forms and tools have
been developed to assist program development
teams with the mapping/alignment of program
outcomes to subject outcomes to assessments.
Both formative and summative program review
processes now include a mapping process which
identifies gaps and redundancies (to support
program updates). Both also include an analysis of
program alignment with Seneca's Academic Plan
(cross-disciplinary,
networked,
experiential
learning; core literacies; teaching excellence;
pathways/partnerships).
PQAPA: 18 Month Follow-up Report
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Review new program development process and forms.
9
Recommendation 4.
The College should consider setting an aggressive schedule for completing all remaining Tier 3 reviews and, regardless of environmental
circumstances, this schedule should be respected and maintained.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
Establish a 7 year review schedule for all college programs and put measures in place to ensure adherence to the schedule.
WHO IS THE LEAD?
Associate Vice President
Academic, Educational
Excellence
WHAT IS THE
DEADLINE?
June 2013
WHAT IS THE CURRENT STATUS?
Include quality assurance goals and measures in Seneca’s Annual
Business Plan.
Completed
Compiled and established a 7 year review schedule for
all college programs. The schedule is set to be reviewed
annually by the Deans’ Group, and will be updated as
required.
Raise profile of status reports by presenting to college councils
and the Board.
Annual quality assurance targets and measures have
been published in Seneca’s Business Plan.
Work plans for Academic Council, Deans’ Group, and
Board of Governors now include regular opportunities
for quality assurance updates and reports.
PQAPA: 18 Month Follow-up Report
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Work with Deans’ Group to establish 7 year schedule to be
reviewed/adjusted annually as needed.
1
0
Table 2: Follow-Up on Affirmations:
TABLE 2: FOLLOW-UP ON AFFIRMATIONS
Affirmation 1.
The College’s commitment of intent to ensuring faculty and students’ awareness of program learning outcomes and the more detailed
mapping of final and major assessments by faculty is encouraging.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
Over the past 14 months Seneca has made great strides towards elevating the awareness and importance of program and subject learning outcomes at the School,
program and individual faculty level. We are ensuring that program learning outcomes are used to guide curriculum design and development, as well as the
assessment of student learning.
Make a concerted effort to increase frequency of workshops
relative to program learning outcomes and alignment of
assessments with outcomes. Offer learning outcome workshops
for chairs and faculty on an ongoing basis to support program
development, review, and teaching practice.
WHO IS THE LEAD?
Associate Vice President
Academic, Educational
Excellence, Centre for
Academic Excellence
(CAE), Deans, and Chairs
Associate Vice President
Academic, Educational
Excellence, Centre for
Academic Excellence
(CAE).
WHAT IS THE
DEADLINE?
Ongoing
Ongoing
WHAT IS THE CURRENT STATUS?
Program learning outcomes are posted on the
college website. Process is in place to monitor
posting.
Over the past year, the Centre for Academic
Excellence (CAE) has facilitated more than 10
professional development workshops focused
specifically on learning outcomes and assessment,
attended by more than 200 faculty and staff.
Learning outcomes workshops offered for chairs
and faculty on an ongoing basis to support
PQAPA: 18 Month Follow-up Report
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Verify all programs with published standards have posted
outcomes on program webpages and continue to post and
confirm program descriptions/goals for all programs without
published program standards.
1
1
program
practice.
Review and revise the Faculty Development Program to elevate
topics related to writing SMART learning outcomes, and aligning
outcomes and assessments with program outcomes.
Associate Vice President
Academic, Educational
Excellence, Centre for
Academic Excellence
(CAE).
December 2012
development,
review
and
teaching
Created a Learning Outcomes Primer to be used by
faculty program development and program review
teams.
Completed.
We have revamped our Faculty Development
Program. It focuses on teaching-related
competencies including curriculum design,
development and assessment, inclusive teaching
and scholarly practice, and technology enhanced
practice. In our Foundations of Teaching and
Learning course, faculty are required to
demonstrate proficiency with writing well written
(SMART) learning outcomes, as well as aligning
outcomes with lesson plans, teaching methods and
assessment strategies.
The College’s recent requirement to add Essential Employability Skills (EES) learning outcomes to all subject outlines.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
Conduct a review of the Subject Outline web system and add functionality to incorporate EES outcomes as appropriate to each subject.
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Create a new subject outline system, which includes Essential
Employability Skills (EES) mapping as required fields.
Align Seneca's Core Literacies (published in Seneca’s 2012-2017
Academic Plan) with EES.
WHO IS THE LEAD?
Academic Faculties and
Schools
WHAT IS THE
DEADLINE?
Fall 2012
WHAT IS THE CURRENT STATUS?
Completed.
Launched a new subject outline system, which
includes EES mapping as required fields.
Completed addition of EES to subject outlines.
Aligned Seneca's Core Literacies with EES and made
it one of the areas of focus for the new Academic
Plan.
PQAPA: 18 Month Follow-up Report
Affirmation 2.
1
2
Affirmation 3.
The College’s recognition that it must either revise the Faculty Performance Review Policy or find ways to help the Chairs adhere to this
requirement.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
Conduct a review of the current Faculty Performance Review policy and, as an interim measure, update the policy to reflect the current practice. Conduct a review the
Faculty Performance Review process, and research best practices to develop a new process that is both meaningful and sustainable.
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Draft Seneca College's Teaching Philosophy and Standards of
Practice and align the Faculty Development Program with the
Teaching Philosophy.
Associate Vice President
Academic, Educational
Excellence
Align the Faculty Performance Review process with the Teaching
Philosophy.
Centre for Academic
Excellence (CAE)
May 2013
Completed
Develop and approve a new Faculty Performance Review policy.
HR, Academic Schools
Early 2014
Scheduled for fall 2013.
Early 2014
Scheduled for early in 2014.
WHAT IS THE
DEADLINE?
January 2013
WHAT IS THE CURRENT STATUS?
Completed
Consultations with appropriate stakeholders are
underway to finalize the draft standards.
PQAPA: 18 Month Follow-up Report
Launch a pilot of a Faculty Performance Review process.
WHO IS THE LEAD?
1
3
Affirmation 4.
The College’s recognition that it needs to provide professional development workshops on writing learning outcomes appropriate to
the credential and orientation sessions early in the cycle to assist in the on time completion of the Report.
WHAT DID THE COLLEGE IDENTIFY AS NEEDING TO BE DONE?
Increase level and frequency of professional development workshops (see Affirmation #1, above) and ensure program review teams are oriented earlier in the cycle.
Continue to focus on and build capacity in learning outcomes
writing, mapping of course learning outcomes to program
learning outcomes, and assessment of learning outcomes.
WHO IS THE LEAD?
Associate Vice President
Academic, Educational
Excellence
Centre for Academic
Excellence (CAE)
WHAT IS THE
DEADLINE?
Ongoing
Program review
orientations to
be held in
December /
early January in
advance of new
round of
summative
review.
WHAT IS THE CURRENT STATUS?
The CAE has developed a Learning Outcomes
Primer to be used by faculty program development
and program review teams in order to write
meaningful and measurable learning outcomes
aligned to assessments. In addition, the CAE has
supported small teams of faculty and
administrators to attend the COU sponsored
symposium on Learning Outcomes thus ensuring
that the CAE and the institution are kept current in
theory and practice.
Over the past year, the Centre for Academic
Excellence (CAE) has facilitated more than 10
professional development workshops focused
specifically on learning outcomes and assessment,
attended by more than 200 faculty and staff.
Writing learning outcomes workshop developed
and implemented for all summative program
review teams.
PQAPA: 18 Month Follow-up Report
WHAT TASKS ARE /WERE ASSOCIATED WITH ADDRESSING THE
RECOMMENDATION?
Develop and implement a Writing Learning Outcomes workshop
for all summative program review teams.
1
4