Quality Assurance
and
Quality Control
Workshop in the framework of the project
"Improving the Labor Market Responsiveness of
Technical Education in Tanzania”,
Ton Vroeijenstijn
Quality Consultants
The Netherlands
Program
1. Do we speak the same language?
2. Towards an Internal Quality Assurance system
3. How to discover the quality of the Institution?
4. How to discover the quality of a program?
5. The IUCEA handbook and the TI’s
Expected learning outcomes
After the workshop, it is expected that you will be able:
 to describe the concepts of “quality” and “quality assurance” in Higher
Education
 to participate in the on-going discussion about quality
 to discover the quality of your institution by means of a self assessment
 To discover the quality of a program
part 1
“Do we speak the same language?”
4
What is Quality?
Basic questions concerning quality
 What is quality?
 Who are the players in the quality game?
 Can quality be measured? (Criteria and standards)
 Who set the standards?
 how do we know the quality?
6
HIGHER EDUCATION
Players in the quality field
 students
 staff
 Government / parliament
 employers/professional bodies
 NACTE
 international forum
7
DIFFERENT VIEWS ON QUALITY
 Quality is the satisfaction of the client
 Quality as excellence
 Quality as additional value
 Quality as value for money
 Quality as fitness for purpose
 Quality as meeting threshold standards
8
Quality
 Absolute Quality does not exist
 There is no fixed definition of Quality
 View on Quality is changing in time
But we must find a workable concept, shared by all
9
Stakeholders and Quality
10
Quality is
• achieving our goals and aims in an
efficient and effective way (fitness for
purpose),
• assuming that the goals and aims
reflect the requirements/expectations of
all our stakeholders in an adequate way
(fitness of purpose).
11
CBE
Quality control and Assurance policy
the term quality refers to “fitness for purpose” (i.e. the
institution and its components of activities have
“quality” if they conform to the purpose for which they
were designed).
Quality
 Is context based
 But there is always a threshold quality
International
standards
University of
the
Amazonas
University
Rio de
Janeiro
Berkeley
National
standards
13
Criteria & standards
 Criterion:
 the specification of elements against which a judgment
is made
 a specific aspect, essential for the quality
 Standard: the level that a criterion must reach. Normally:
adequate or satisfactory. Sometimes quantifiable
14
Criteria & standards
 Who is setting standards?
 Who is checking the standards?
 Because there is no absolute quality, there are no
absolute standards; standards are a matter of
negotiation.
 Criteria: valid in all circumstances
 Standards: context bound: Mc Donald and 3 stars
restaurant
15
Can Quality be measured?
 Tendency of governments to quantify quality aspects
 Use of performance indicators?
 But….performance indicators does not tell us all about
the quality
 “Performance indicators may harm the quality”
16
Performance indicators and quality assessment
Acidity
% alcohol
tannin
sediment
performance
indicators
smell
taste
expert team
17
Without quality assurance, no TI will survive
QA in Industry
 TQM-Deming cycle
 ISO
 EFQM
Quality Assurance in HE
 a process of establishing stakeholder confidence that
provision (input, process and outcomes) fulfils
expectations or measures up to threshold minimum
requirements. (Harvey, 2004, Analytic Quality Glossary)
 Systematic, structured and continuous attention to quality
in terms of maintaining and improving quality. (
IUCEA handbook A Roadmap to quality)
Why Quality assurance is needed
 More students, less money
 Rise of private institutions
 Growing competition
 Changing expectations labour market
 Globalisation
 Tax payer want value for money
 “ Consumer” protection
Quality management in Higher Education
•
•
•
•
• Formal EQA
Accreditation
Informal External QA
Audit
Assessment
benchmarking
Internal
QA
planning
monitoring
:
improvement
evaluation
Internal Quality Assurance
IQA is:
Systematic, structured and continuous attention
to quality in terms of maintaining and improving
quality in research, community service,
instruction
Plan
Act
Do
Check
Key-questions
 Do we do the right things?
 Do we do the right things in the right way?
 Do we have a thorough command of the process to realize
actually what we want?
 Do we really achieve what we want to achieve?
24
What instruments do we have for
IQA??
 Monitoring systems:
 Student progress
 Pass rates/drop outs
 Structural contacts employers
 Structural contact alumni
 Evaluation instruments
 Student evaluation
 Course evaluation
 Curriculum evaluation
 Improvement instruments:
 Staff development activities
 Curriculum design
 SWOT-analysis/self assessment
 External assessment
Implementation IQA-system

there is no one fit for all

IQA system is tailor made, but ….

Requirements set by out side agencies

Code of good practice ENQA, INQAAHE, UNESCO
26
CRITERIA FOR IQA
(enqa)
1
Policy and procedures for QA
2
Monitoring system
3
Periodic review of the core activities
4
Quality assurance of the student assessment
5
Quality assurance of staff
6
Quality assurance of facilities
7
Quality assurance of the student support
8
Self assessment
9
Internal audit
10
Information systems
11
Public information
12
A Quality handbook
27
Basic conditions for IQA
 As simple as possible and not bureaucratic
 Balance between centralized and decentralized
approach
 Supported by management
 Effective instruments
 Tuned to national and international developments
28
Problems with the introduction of IQA
 Lack of quality awareness
 Resistance against innovations
 Resistance of staff because they feel threatened
 Lack of knowledge about QA. Training is needed.
 resistance because it is time consuming and money consuming
(“We have other things to do”).
 It is difficult to define what quality is; the QA indicators are not
always clear;
 The purpose and the added value are not always clear
 Lack of clear communication between the staff and the
institutions management
29
To overcome the problems it is important to:
 Understand clearly what Quality Assurance means;
 Know the available instruments;
 Know about the requirements set for an IQA system
 Design the system very clearly and formulate the strategy to
introduce it
 Tune the system to external developments.
30
1. QA-0rganisation and procedures
1.1. QA-unit & committees
1.2. Quality(improvement plan)
2
Student
Progress
3
Pass rates
Drop out rates
4
Feedback Labour
market
&
Alumni
Evaluation
instruments
6
Student
evaluation
7
Course/curriculum
evaluation
8
Research evaluation
Special QA
processes
10
Assurance
Quality
student
assessment
11
Assurance
Quality
staff
12
Assurance
Quality
Facilities &
infrastructure
Monitoring
instruments
Specific QA
instruments
14
SWOT-analysis
Self assessment
15
Inter-collegial audit
or
Peer review
16
Information systems
18. Follow up activities
5
Research
Performance
9
Community
Service
evaluation
13
Assurance
Quality
Student support
17
Quality handbook
Self-assessment: a powerful
instrument
 Analysis of the quality of the
institution/faculty/department/program
 Promotion of quality awareness and strengthening the
”corporate identity”
 Providing information to the external expert team
CONDITIONS FOR SELF ANALYSIS
A self assessment:
 should never be felt as threatening
 aims at improvement and enhancement
 need a broad support; Everybody has to be involved.
 Need support of the management
 demands a good organisation.
 is an analysis supported by the whole faculty
 Not everyone has to agree with all the points in the selfassessment report.
Part 3
How to discover the quality of the
institution?
1. Requirements stakeholders
3
Policy
plan
2
Mission
Vision
4
Leadership
&
administration
5
Human
resource
•
•
•
•
•
•
•
8
Educational
Effectiveness
The program
Student
assessment
Staff
Student admission
Facilities
Infrastructure
Partnership HEI’s
10
benchmarking
Aims
Objectives
6
Funding
&
Financial
management
7
Facilities
&
Infrastructure
11
Quality
assurance
9
Community
contribution
13. Satisfaction stakeholders
12
A
C
H
I
E
V
E
M
E
N
T
S
Academic standards NACTE
Standard 1: Institutional Vision and Mission;
Standard 2: Governance and Administration;
Standard 3: Institutional Integrity;
Standard 4: Institutional Effectiveness;
Standard 5: Educational Programmes;
Standard 6: Student Information and Admission to Programmes;
Standard 7: Student Guidance and Support;
Standard 8: Staff Selection, Appraisal and Development;
Standard 9: Physical Resources; and
Standard 10: Financial Resources.
Quality criteria at Institutional level
IUCEA
•Philosophy, Mission, vision, of the institution
•Policy plan
•Governance
•Human Resource
•Funding and Financial Management
•Educational activities
•Research
•Community outreach
•benchmarking
•Quality assurance
•Achievements/outcomes
•Satisfaction of the stakeholders
36
1. Requirements stakeholders
3
Policy
plan
2
Mission
Vision
4
Leadership
&
administration
5
Human
resource
8
Educational
Effectiveness
•The program
•Student assessment
•Staff
•Student admission
•Facilities
•Infrastructure
•Partnership HEI’s
10
benchmarking
Aims
Objectives
6
Funding
&
Financial
management
7
Facilities
&
Infrastructure
11
Quality
assurance
9
Community
contribution
13. Satisfaction stakeholders
12
A
C
H
I
E
V
E
M
E
N
T
S
part 4
How to discover the quality of
our programs??
 A program is defined as a coherent set of courses
leading to a certain degree (bachelor or master).
 We may call the program also a curriculum
 It is not only about the content, but also about the
process and the boundary conditions
Program/ curriculum
I nstitutional
capacity
1.
Mission
&
objectives
2
Leadership
&
Administration
3.
Financial
Resources
5
E
X
P
E
C
T
E
D
L
E
A
R
N
I
N
G
O
U
T
C
O
M
E
S
4.Requirements stakeholders
Process
Input
Quality
Assurance
6
Programme
specificatio
n
7
Content of
the
programme
8
Organization
the
programme
9
Didactic
concept
11
Quality
Academic
staff
12
Quality
Suppor t
staff
13
Profile
Of
students
14
Student
Advice
&
suppor t
16
Student
evaluation
17
Curriculum
design
18
Staff development
activities
10
Student
Assessment
15
Facilities
&
Infrastructu
re
19
Benchmarking
21.Satisfaction stakeholders
22.Par tnership with HEI’s
23
Research
24
Community service
20
Achievements:
The graduates
Criteria to be assessed
•Goals and objectives; expected learning outcomes
•Program content
•Program specification or description
•Program organisation
•Didactic concept/teaching/learning strategy
•Student assessment
•Staff quality
•Quality of the support staff
•Student profile
•Student advice/support
•Facilities & infrastructure
•Student evaluation
•Curriculum design & evaluation
•Staff development activities
•Benchmarking
•Achievements /graduates
•Satisfaction stakeholders
Program/curriculum
Institutional
capacity
1.
Mission
&
objectives
2
Leadership
&
Administration
3.
Financial
Resources
5
E
X
P
E
C
T
E
D
L
E
A
R
N
I
N
G
O
U
T
C
O
M
E
S
4.Requirements stakeholders
Process
Input
Quality
Assurance
6
Programme
specification
7
Content of
the
programme
8
Organization
the
programme
9
Didactic
concept
11
Quality
Academic
staff
12
Quality
Support
staff
13
Profile
Of
students
14
Student
Advice
&
support
16
Student
evaluation
17
Curriculum
design
18
Staff development
activities
10
Student
Assessment
15
Facilities
&
Infrastructur
e
19
Benchmarking
21.Satisfaction stakeholders
22.Partnership with HEI’s
23
Research
24
Community service
20
Achievements:
The graduates
Strengths and weaknesses analysis
1
1
Goals and objectives; expected learning outcomes
2
Programme content
3
Programme specification
4
Programme organisation
5
Didactic concept/teaching/learning strategy
6
Student assessment
7
Staff quality
8
Quality of the support staff
9
Student profile
10
Student advice/support
2
3
4
5
6
7
x
x
x
x
x
x
x
x
x
41
The 7-point scale read as follows
 1= absolutely inadequate; immediate improvements
must be made
 2= inadequate, improvements necessary
 3= inadequate, but minor improvements will make it
adequate
 4= adequate as expected
 5= better than adequate
 6= example of good practice
 7= excellent
The follow-up of the self-assessment
 In many cases followed by accreditation
 If not: organize an inter-collegial assessment
 Outcomes of the SWOT-analysis is used for a quality
plan
 Each year check of the quality plan
Impact
 Does al efforts put in Quality Assurance have impact?
 Difficult to measure
 Different views on quality
 QA is one of the activities effecting HE
 Methodological problem: no comparison how it was
before
 “Students do not perform better”
 There are changes, but “we had already planned this
before.
Some changes
 Shift from research oriented to education oriented
 Individual courses ------ curriculum as a joint enterprise
 Teacher oriented --------student oriented
 Better structured curricula; learning outcomes
 People start to think about qulity
Negative
 Teacher perceive is as a bureaucratic burden
 Compliance culture
 Hindering innovation
 Ritual dance
 Window dressing
Part 5
The IUCEA handbook
Why an IUCEA quality handbook was
seen as needed
Development in the region of
 a shared idea of quality
 harmonized internal quality assurance systems
 a shared idea about criteria and standards
 shared ideas about the accreditation framework
A Road Map to quality
 Initiative of the IUCEA 2006, supported by DAAD and HRK of Germany
 Based
 on documents and experiences of the national regulatory bodies in East Africa
 Based on similar developments in South-East Asia, Central America and
Europe
 Editorial board with members of the standing committee of the IUCEA
 Discussion of drafts at the different workshops
 5th draft used by the 1st cohort of universities
 Revised draft (4 volumes ) by 2nd cohort
 Endorsed by IUCEA and official publication June 2009 (www.iucea.org)
Contents of the handbook
 Volume 1
Guidelines for Self- assessment at program level aims at the
faculty/department offering an instrument to learn more about the quality of
the programs at offer by means of an effective self assessment at program
level
 Volume 2:
Guidelines for external assessment explains the procedures and processes
for an external assessment at program level. The specific target group is the
external expert team, but also the faculty/department to be assessed.
 Volume 3:
Guidelines for Self-assessment at institutional level aims especially at the
central management of an institution and offers an instrument to discover
more about the quality of the institution
 Volume 4:
The implementation of a Quality Assurance system aims at all level of an
institution, but is especially useful for the Quality Assurance coordinators
for the development and installation of an Internal Quality Assurance
system
How to use the handbook
1. The handbook is not forcing an institution to apply
fixed standards and ideas
2. It is voluntary, but…….. members of IUCA should
play the rules of the game
3. For private institutions: it will help to be in line with
international developments
4. The handbook offers a toolkit for quality
assurance and quality improvement.
1.
Central management can use the handbook to discuss and implement a
quality policy
2.
The Quality-officer can use the handbook to promote quality assurance
in the institution and to facilitate the quality process in the university. The
handbook contains basic materials for training sessions.
3.
Faculty deans can use the instrument for self assessment at program
level and provide staff and students with basic background information
4.
Staff and students can use the handbook for better understanding of the
need of quality assurance and also for a better understanding of the
instruments.
.
Some statements
1.Responsibility for quality assurance should be
replaced: more efforts from the TI’s, less from
NACTE
2.It is advisable to develop a handbook for quality
assurance, based on the Roadmap, to be used
by the TI’s
3.NACTE will bring its process,
procedures and
.
publications in line with the handbook