Accreditation Visit to
<name of HEI>
<date range>
Name
Title
The road to a P.ENG. begins with the right education
P. ENG. *
Welcome!
• Outline of this presentation
•
•
•
•
•
•
•
•
•
•
•
•
Background
General notes on accreditation
Goals of the CEAB
Objectives of the visit
Team’s responsibilities
Tasks and tools
Visit schedule
Accreditation criteria highlights
NEW! Graduate attribute criteria
Potential issues
Post-Visit Activities
Visit-Specific Issues
Background
• History and current situation
• <when HEI opened>
• <current student complement>
• <any major contextual factors>
• Purpose of this visit
• <list programs being visited>
• <indicate when these programs were most recently
visited and what the CEAB decisions was>
General notes about
accreditation
• Accreditation applies only to programs, not to
departments or faculties
• Undertaken only at the invitation of the HEI and with
the consent of the appropriate regulator
• Accreditation constitutes:
• Quantitative and qualitative evaluation of the curriculum
• Qualitative evaluation of the program environment
• Accreditation is granted for a period of time up to and
normally not exceeding six years
Goals of the CEAB
• Engineering programs offered by Canadian
institutions will meet or exceed minimum educational
standards acceptable for professional engineering
licensure in Canada
• The quality and relevance of engineering education
will continuously improve
• The Engineers Canada Board of Directors will be
provided with advice and recommendations on
international matters relating to engineering
accreditation and education
Objectives of the visiting team
• Conduct fact-finding on behalf of the Accreditation Board
• Review, validate and/or add to the information provided by the
host institution
• Review of materials, meetings, and facility tours to corroborate
program strengths and weaknesses and bring forward issues to
the CEAB
• Describe progress toward use of graduate attributes in program
assessment and improvement
• Collaborate in preparing a report of the team’s findings
• The visiting team or its members do not make any
recommendations ~ accreditation decisions are made by
the CEAB
Team’s responsibilities
• Thoroughly read the institution’s completed
questionnaire
• Identify issues for investigation during the visit
• Attend the visit
• Participate in team discussions
• Complete your report (before the exit meeting!)
• Be available to answer questions after the visit and
before the CEAB’s decision meeting
Team’s responsibilities ~ cont’d.
•
The confidential nature of the process
•
Check everything you question with the program coordinator or other responsible
person ~ don’t assume!
•
Be sure to get the names of all the people that you interview. Circulate an
attendance list.
•
Visitors should not get into giving suggestions
•
All issues that are to appear in the final report must be brought up in the exit
meeting ~ i.e. there must not be any surprises in the final report.
•
We must agree on all issues to be raised and I must have a clear understanding of
them.
•
Do not use the terms “concern”, “weakness” or “deficiency”. In the accreditation
process, these terms have very specific meanings (accreditation jargon!).
Tasks and tools
•
Interviews with appropriate senior administrative officers, including the
president, the dean of engineering and the chairs of the departments
responsible for the programs
•
Interviews with individuals and groups of faculty members to evaluate:
• professional attitudes
• motivations
• morale
• the balance of opinions concerning theoretical and practical
elements of the curriculum
Interviews with individuals and groups of students. Ask open-ended
questions to get them talking
•
•
Examine compliance with graduate attribute criteria
Tasks and tools ~ cont’d.
• Tours of physical facilities such as laboratories, libraries, and
computing facilities, to evaluate their effectiveness
• Note that the Accreditation Board does not require any Faculty
to spend money - the question is whether the equipment,
supplies, etc. are adequate
• A review of recent examination papers, laboratory instruction
sheets, student transcripts, student reports and theses, models
or equipment constructed by students and other evidence of
student performance
• Are performance expectations and grading standards
appropriate?
Tasks and tools ~ cont’d.
• Attendance form ~ for meetings and interviews
• AU re-allocation form
• Sample questions for faculty and staff
• Sample questions for students
• Interpretive Statements and Regulations
• Interpretive statement on natural sciences
• Interpretive statement on licensure expectations and requirements
• Regulations for granting transfer credits (effective January 2012)
Visit Schedule
Pre-visit Activities
• All team members review the written material
submitted by the host institution, and spend time
reviewing graduate attributes
• Team members prepare a list of potential issues and
a preliminary draft of their contribution to the report.
• Initial team meeting
• A detailed schedule for Sunday to Tuesday has been
prepared for each team member
Visit Schedule ~ cont’d.
Visit – Day 1 (Sunday)
• Lunch with institution officials
• In the afternoon, the team visits the institution to
review course materials including consideration of
graduate attribute compliance
• Team dines together
• Team meets in the evening ~ agenda includes
discussion of:
• observations and findings of the day
• potential issues and how to investigate further
• previous decision issues and areas to be re-examined
Visit Schedule ~ cont’d.
Visit – Day 2 (Monday)
• Plenary Session with the Team, Dean, Program Chairs / Co-ordinators:
• Introductions and purpose (fact-finding for the CEAB)
• Process and time lines
•
Conduct visit
•
Lunch with administrators and faculty members
•
Continuation of visit
•
Team dinner and meeting
• Discussion and preliminary consensus regarding issues
• List areas of strength and list issues that require further
investigation
Visit Schedule ~ cont’d.
Visit – Day 3 (Tuesday)
• Update information with Dean, Program Coordinators
• Revise visit schedule as necessary
• In camera Team Working Lunch:
• Complete draft reports
• Review reports, arrive at consensus on final conclusions
• Draft copies of each report provided to the chair
Visit Schedule ~ cont’d.
Visit – Day 3 (Tuesday) ~ cont’d.
• Exit meeting with Dean, Program Chairs, Faculty
(verbal presentation by the Team Chair)
–
–
–
–
–
–
Restate - Visiting Team’s role is fact-finding
Accreditation decisions are made only by the Accreditation Board
Repeat time-line of the process to follow
Emphasize confidentiality
Summarize all issues and state strengths
Thank the Dean and staff for visit arrangements and their
hospitality
Timeline after visit
- Chair submits report to CEAB Secretariat
- Report is edited, formatted and returned with any
questions to chair
- Chair may contact team members with questions
- Report finalized, sent to institution
- Institution responds and sends update
- Accreditation decision made (June or Sept mtg)
- Institution and Team members notified of decision
(within month)
If you see an issue with a
program
- Visit Team is on a fact-finding mission
- Institution’s documentation will emphasize the
positive but your direct observation may differ
- You need to verify documentation and identify
discrepancies if any, to inform CEAB decision
- Add something about editing process.
- If there is an issue, the institution still has multiple
opportunities to address it and improve
- Do not hesitate to dig for the full picture and describe
it accurately in your report
Accreditation Decisions
2013-2014 Cycle
25
20
15
10
5
0
6V
3R
4V
3V
2R
1R
1V
AM
AD
There are now 278 accredited programs at 43 Higher Education
Institutions in Canada
And 14 substantially equivalent programs in 3 HEIs outside of Canada
Accreditation Criteria Highlights
• The processes of accreditation place emphasis on
the quality of the curriculum, the students, the
academic staff, support staff, facilities, and resources
• The accreditation criteria:
• Reflect the need for engineers to be adaptive, creative, resourceful,
and responsive
• Ensure that the graduates understand the role and responsibilities of
professional engineers to society
• Reflect the need for the professional engineer to function as an
effective member of a team and to communicate effectively
Accreditation Criteria Highlights
cont’d.
Qualitative Evaluation – Curriculum Considerations
• Curriculum must include the application of computers and
appropriate laboratory experience and safety procedures
• Students must be exposed to material dealing with
professionalism, ethics, equity, public and worker safety and
health considerations, concepts of sustainable development,
environmental stewardship
• The Curriculum must prepare students to learn independently
and to work as an effective member of a team
Accreditation Criteria Highlights
cont’d.
Qualitative Evaluation – Curriculum Considerations
• Curriculum must include studies in:
– communication skills
– engineering economics
– impact of technology on society
– subject matter that deals with central issues, methodologies
and thought processes of humanities and social sciences,
and;
– must culminate in a significant design experience
Accreditation Criteria Highlights
cont’d.
Qualitative Evaluation – Curriculum Considerations
• Engineering Design:
– integration of curriculum elements
– creative, iterative and open-ended
– subject to constraints imposed by legislation or standards
– to satisfy specification using optimization
– economics should be part of the design experience
– to be supervised by licensed engineers
Every program must culminate in a significant design
experience
Accreditation Criteria Highlights
cont’d.
Quantitative Evaluation
Accreditation units (AU)
• one hour of lecture (corresponding to 50
minutes of activity) = 1 AU
• one hour of laboratory or scheduled tutorial =
0.5 AU
Accreditation Criteria Highlights
cont’d.
Accreditation Criteria Highlights
cont’d.
• In reviewing course information and course
materials, check reasonableness of AU allocations ~
not an exact science!
• AU re-allocations should be team decisions, after
discussion
• We will discuss re-allocations, if any, on Sunday evening, and
again on Monday evening
• Discuss allocations with responsible faculty member,
but no need to argue
• Agree to disagree
Accreditation Criteria Highlights
cont’d.
Professional Licensure
• Dean, Department Chairs, and faculty members teaching
courses that are primarily engineering science and engineering
design are expected to be licensed to practice engineering in
Canada
• minimum of 225AU of ED to be instructed by P.Eng./ing.
• minimum of 600 AU of ES+ED to be instructed by P.Eng./ing. or
'Applied’
• Interpretive Statement as guidance
• Curriculum development and control should be in the hands of
persons licensed to practice engineering in Canada
Graduate Attributes
Accreditation Criteria Highlights
cont’d. – Graduate Attributes
Criterion 3.1 Graduate Attributes
• Two components
• Attributes:
– Interpreted at time of graduation
– Recognized that achievement does not end there
• Continuous Improvement:
– Ongoing evolution of engineering programs
– Processes needed
» Assessment of attribute achievement
» Results used to improve program
Graduate Attributes
Accreditation Criteria Highlights
cont’d. – Graduate Attributes
Criterion 3.1 Graduate Attributes
Timeframe:
• “While programs are expected to provide evidence to
demonstrate compliance with this criterion, a
transition and development period will be allowed.
Starting in June 2015, the Accreditation Board will
make decisions about compliance with the Graduate
Attribute criteria. Deficiencies may be assessed in
cases of non-compliance.”
Graduate Attributes
Accreditation Criteria Highlights
cont’d. – Graduate Attributes
1. Knowledge Base
7.
Communication
2. Problem Analysis
8.
Professionalism
3. Investigation
9.
Impact on Society and
the Environment
4. Design
10. Ethics and Equity
5. Use of Engineering
Tools
11. Economics and Project
Management
6. Individual and Team
work
12. Life-Long Learning
Graduate Attributes
Graduate Attributes ~ Examples
The institution must demonstrate that the graduates of a program
possess the attributes under the following headings:
3.1.1 A Knowledge Base for Engineering: Demonstrated
competence in university level mathematics, natural sciences,
engineering fundamentals, and specialized engineering knowledge
appropriate to the program.
3.1.8 Professionalism: An understanding of the roles and
responsibilities of the professional engineer in society, especially
the primary role of protection of the public and the public interest.
Graduate Attributes
Graduate Attributes progress ~
Evaluation by Program Visitors
The Program Visitor must evaluate progress toward each
graduate attribute to fill out the report template:
Program Visitor’s Observations on Implementation
Evaluate the evidence and actions (either seen on-site or in the
questionnaire) proposed to demonstrate the level of achievement
of each graduate attribute
Program Visitor’s Observations on Implementation
Evaluate the evidence and actions (either seen on-site or in the
questionnaire) proposed to demonstrate the level of continual
improvement achievement
Graduate Attributes
Graduate Attributes progress ~
Evaluation by Program Visitors
• Things the Program Visitor will need to see:
– Graduate Attributes (Accreditation Criteria)
– Learning outcomes that support Graduate
Attributes
– Indicators
– Acceptable levels
– Feedback mechanism
Graduate Attributes
Graduate Attributes:
Evaluation by HEI

The program is assessed, not the students

Continuous improvement process

Not required to assess every student

Not required to assess in every course

Not required to assess every year
Graduate Attributes
Continuous Improvement –
The big picture
HEIs evaluate:
• Are students meeting expectations?
– In what areas are they successful
– What areas require improvement
• What data would help institution improve their
program?
Graduate Attributes
What to look for:
Program background
• Is the program clearly described?
– Is there a curriculum map?
• Is the context of the program clear?
Graduate Attributes
What to look for:
Curriculum Mapping
• Information in the curriculum map is
– Accurate, with some depth
– Identifies intended outcomes from learning
experiences
– Not simply a list of topics “covered”
• Map provides information for each attribute
– Can include curricular and other experiences
Graduate Attributes
Methodology: Data Collection
Plan
• On what does the program propose collecting
data (i.e. indicators)?
• What methods are proposed for collecting
data?
• Is the data collection plan good?
Graduate Attributes
Terminology for Data Collection
(1)
•
•
•
•
Valid Indicators
measure what they are supposed to
measure
Reliable Indicators
the results are consistent; the measurements
are the same when repeated with the same
subjects under the same conditions
Graduate Attributes
Terminology for Data Collection
(2)
Direct measures
•
directly observable or measurable
assessments of student learning
Indirect measures
•
opinion or self-reports of student learning or
educational experiences
Use both direct and indirect measures if
possible.
Graduate Attributes
Data Collection - Indicators
• An indicator is like a sensor: what indicators
has the program chosen?
• Where have they placed their indicators?
Where are the data collection points?
• Does the proposed data collection plan make
sense?
Graduate Attributes
What to look for:
Overall - data collection plan
• Integrity:
– How good is the quality of the data collection plan
• Are Indicators well chosen?
• Are assessment points well chosen?
– Is valid, reliable data collection proposed?
– Is plan cyclic, continuous?
• will results be useful for informing curriculum
change? Ask the question: “why are you
collecting this data?”
Graduate Attributes
What to look for:
Indicators in data collection
• Indicators align with attributes and questions
• Indicators are “leading indicators”:
central to attribute; indicate competency
• Enough indicators defined to identify strength areas and
weak areas within an attribute
• Not too many indicators – resulting in reams of data but
little deep information
• Indicators are clearly articulated and measurable
Graduate Attributes
Selecting Assessment Points
• Learning is generally demonstrated
through:
– Artifacts, e.g. written test, report, built project
– Performances, e.g. oral presentation,
observed practice
Graduate Attributes
What to look for:
Assessment Points
• Indicators are well aligned to the proposed
assessment points
• Enough assessment points are utilized
• Expectations of performance quality are
clear, i.e. the scale is defined
Graduate Attributes
What to look for: Triangulation
in improvement process
• Are opportunities included for informal assessment,
students’ self-reports of learning, and even unsolicited
data from placement supervisors or employers?
• Are more than one type of assessment used when
analyzing data?
• Are all assessments valued, not just major events?
• Are the data gained from assessment used to answer
questions about authentic learning?
• Are data across time intervals looked at?
Graduate Attributes
Measurement Tools ~
Illustrative Examples
•
Examinations
–
–
–
–
–
–
•
–
–
–
–
–
–
Final
Mid-term
Entry and exit
Standardized (PPE, FE)
Oral
Embedded questions
•
•
Portfolios
–
–
–
–
–
Culminating design experience
Projects
Laboratories
Internship/stage
Co-op
Surveys
Exit
Alumni
Employers
Self
Course Evaluations
Advisory Board
Student Work
–
–
–
–
–
Reports
Peer Reviews
Reviews/critiques
Presentations
Posters
Graduate Attributes
Quality Improvement Loop
• 3.1 Graduate attributes
– Engineering programs are expected to continually improve
– There must be processes in place that demonstrate that program
outcomes are being assessed in the context of the attributes
– And, that the results are applied to the further development of the
program.
Thus,
If observed outcomes are not consistent with
expected attributes, then system inputs and/or
process must be adjusted
Graduate Attributes
What to look for:
Evidence of Feedback Loop
• Results are consolidated for each learning outcome.
• Determination is made regarding whether learning
outcome is met.
• Results of assessment are used to determine if
changes need to be made in curriculum, courses,
prerequisites, performance criteria or metrics.
• Change is implemented
• Assessment is repeated to determine effect of
change.
Graduate Attributes
Continuous Improvement
Data
Collection
and
Evaluation
Program
Objectives
and
Expected
Outcomes
Assessment
Processes
and
Metrics
Curriculum
Graduate Attributes
Criterion 3.1 ~ 2009 Findings
• Most programs at beginning stages of developing GA
measurement tools and data management systems
• Some examples:
• Mapping of GAs to curriculum
• Structuring activities to address specific GA aspects
Graduate Attributes
Criterion 3.1 ~ 2011-2012 Cycle
Findings
• Accreditation site visits in 2011-2012 assessed state
of progress toward Graduate Attribute compliance.
• Out of 17 institutions, based on present progress:
– 7 were rated “will fail to reach compliance by 2014”
– 3 were rated “likely to fail to reach compliance by
2014”
– 7 were rated “on track to comply by 2014”
Criterion 3.1 and Criterion 3.2 ~
2012-2013 Cycle Findings
• Accreditation site visits in 2012-2013 assessed state
of progress toward Graduate Attribute compliance.
• Out of 15 institutions, based on present progress:
– 6 were rated “not likely to be in compliance with
Criteria 3.1 and 3.2 by 2014”
– 2 were rated “likely to be in compliance with
Criteria 3.1 and 3.2 by 2014”
– 7 were rated “on track to comply with Criteria 3.1
and 3.2 by 2014”
Criterion 3.1 and Criterion 3.2 ~
2013-2014 Cycle Findings
• Accreditation site visits in 2013-2014 assessed state
of progress toward Graduate Attribute compliance.
• Out of 14 institutions, based on present progress:
– 6 were rated “not likely to be in compliance with
Criteria 3.1 and 3.2 by 2014”
– 7 were rated “on track to comply with Criteria 3.1
and 3.2 by 2014”
Potential Issues that visit can
reveal
• Large classes
• Faculty numbers ~ “faculty who are not faculty”
• Long-term leaves and long sabbaticals (counted at institution,
but not available to teach)
• Soft-funded faculty
• Teaching loads ~ critical dependence on a single individual
• Course failure rates
• Check the course information sheet to see if the numbers make
sense (tough courses should have higher failure rates)
• Students pass while failing
• Attrition rate
Potential Issues cont’d.
• Grading of major written reports
• Appropriateness of AU allocations ~ especially for
Engineering Design and Complementary Studies
• Plans for renewal of equipment
• Design experience
• Access to Dean’s office, Program Director's office, etc.
• Notional contact hours
• Admissions:
• Practices followed for granting advanced standing or transfer
credits?
• Who authorizes exceptions?
• Is control rigorous?
Report Expectations
• All issues on the visit
• Must be tied to criteria
• Finalized by Monday night
• Complete report to extent possible by Tuesday p.m.
• Will be used to make exit statement
• Recall ~ all issues need to be raised at the exit meeting
• Your reports transferred to my computer before you
go
Post-Visit Activities
• Chair prepares a draft team report
• Incorporates Program Visitors' reports
• Target ~ within 2 weeks of Visit
• Distributed by e-mail to all team members for review
and comment
• Target ~ comments returned within 4 weeks of Visit
• Report edited by CEAB Executive Committee
member
• Edited report sent to HEI for review and comment
• CEAB relies on report + HEI’s response to make
decision
Post-Visit Activities cont’d.
• All team members will be advised of decision
• Opportunity to be acknowledged by CEAB for your
contribution
• Evaluation
• HEI evaluates the visit process
• Team Chair evaluates team members
• Team members evaluate Team Chair and process (currently
this is a pilot project. Not all teams do this)
• General Visitor provides report on visit process to association
of jurisdiction where institution is located
Visit-Specific Issues
• Insert per pre-visit summary of issues list
For more information:
1100-180 Elgin Street, Ottawa, Ontario K2P 2K3
Tel. 613-232-2474 / Fax. 613-230-5759
engineerscanada.ca
*The term PROFESSIONAL ENGINEER is an official mark owned by Engineers Canada.
The road to a P.Eng. begins with the right education
P. ENG. *