SELF-EVALUATION REPORT OF THE
QUALITY ASSURANCE AGENCY
FOR HIGHER EDUCATION
EXTERNAL REVIEW FOR CONFIRMATION
OF FULL MEMBERSHIP OF ENQA
March 2013
QAA 516 03/13
© The Quality Assurance Agency for Higher Education 2013
ISBN 978 1 84979 823 5
All QAA’s publications are available on our website www.qaa.ac.uk.
Registered charity numbers 1062746 and SC037786
CONTENTS
Foreword 2
Executive summary
3
Introduction 4
About QAA
11
QAA's main areas of work in the UK
14
QAA Strategy 2011-14
20
QAA progress since the 2008 ENQA review
22
QAA's compliance with European Standards and Guidelines (Part 2)
25
25
28
29
30
33
35
36
37
Standard 2.1 Use of internal quality assurance procedures
Standard 2.2 Development of external quality assurance processes
Standard 2.3 Criteria for decisions
Standard 2.4 Processes fit for purpose
Standard 2.5 Reporting
Standard 2.6 Follow-up procedures
Standard 2.7 Periodic reviews
Standard 2.8 System-wide analyses
QAA's compliance with European Standards and Guidelines (Part 3)
Standard 3.1 Use of external quality assurance procedures for higher
education (ENQA membership criterion 1)
Standard 3.2 Official status (ENQA membership criterion 2)
Standard 3.3 Activities (ENQA membership criterion 1)
Standard 3.4 Resources (ENQA membership criterion 3)
Standard 3.5 Mission statement (ENQA membership criterion 4)
Standard 3.6 Independence (ENQA membership criterion 5)
Standard 3.7 External quality assurance criteria and processes used by the agencies (ENQA membership criterion 6)
Standard 3.8 Accountability procedures (ENQA membership criterion 7)
40
40
41
42
43
47
48
50
52
Compliance with ENQA membership criterion 8 (miscellaneous)
56
Development of the QAA self-evaluation report
57
QAA: current challenges and areas for future development
61
Additional reference materials
63
Glossary64
Acknowledgements64
FOREWORD
Since QAA was reviewed by ENQA (the European Association for Quality
Assurance in Higher Education) for the first time in 2008, the world of
higher education quality assurance has faced a period of continuous
change and challenge, both in the UK and across the globe.
In preparing for our second review in 2013, we have been very conscious
of the developments which have taken place over the last five years in
response to these challenges.
I believe there have been significant areas of progress for QAA, notably in
student engagement, improved public information and the development of new review methods.
During this period, QAA has also played an increasingly wider role in international quality assurance
and policy developments. Our membership of ENQA - which plays a unique role in its stewardship
of quality and standards in the European Higher Education Area - and also of INQAAHE
(the International Network for Quality Assurance Agencies in Higher Education) have been
important factors in enabling us to do this. Today, we also continue our programme of international
engagement with agencies around the world, developing relationships and agreements in response
to the growing internationalisation of higher education.
The process of developing this self-evaluation report has proven to be an invaluable developmental
activity for QAA, and will provide a strong platform on which we will continue to build and enhance
our work in future years.
We commend this report to the Board of ENQA and the review panel, and hope that it provides
a full and clear picture of the current work of QAA and its compliance with the Standards and
Guidelines for Quality Assurance in the European Higher Education Area.
Anthony McClaran
Chief Executive
Quality Assurance Agency for Higher Education
2
EXECUTIVE SUMMARY
This report sets out the information and evidence which demonstrates how the UK Quality
Assurance Agency for Higher Education (QAA) meets and complies with the Standards and
Guidelines for Quality Assurance in the European Higher Education Area (ESG), as published by ENQA.
QAA was founded in 1997, and its mission is to safeguard standards and improve the quality of
UK higher education. QAA covers the four nations of the UK - England, Northern Ireland, Scotland
and Wales. It meets the diverse needs of the different higher education policies determined by each
nation, but itself operates as a single entity across the whole of the UK.
QAA employs a number of review methods for different types of UK higher education providers,
including universities, university colleges, further education colleges and alternative providers,
as well as professional programmes and international provision.
The central UK quality assurance reference point is the UK Quality Code for Higher Education
(the Quality Code). The Quality Code sets out the expectations that all UK higher education
providers are required to meet. QAA works closely with the UK higher education sector to develop,
maintain and update the Quality Code. The Quality Code is set out in three Parts:
l
Part A: Setting and maintaining threshold academic standards
l
Part B: Assuring and enhancing academic quality
l
Part C: Information about higher education provision.
The Quality Code underlines QAA's belief that all providers of higher education in the UK should be
quality assured under a common framework which can be adapted in its application in different UK
nations, and which also recognises the value of enhancement.
As an agency, QAA is appropriately resourced to deliver all of its external work programmes to a
high standard, with strong internal quality assurance and accountability processes.
In addition to demonstrating QAA's compliance with the ESG, this report also sets out a number
of features of good practice in the Agency's work. These include the process for developing
the Quality Code, QAA's reviews of international provision/transnational education, the
approach to training QAA reviewers, work on student engagement, and social media and
internet communications.
Following the 2008 review of QAA by ENQA, the Agency has paid close attention to the
recommendations made. QAA has made good progress in providing more accessible information
for the public, in the inclusion of students on its review teams, in establishing more detailed
protocols with its funding councils, and in engaging more stakeholders in the development of this
self-evaluation report. QAA has also made significant steps towards the inclusion of international
reviewers in its review activities.
Looking to the future, QAA envisages positive developments for the Agency as it continues to
manage quality assurance in higher education across the four nations of the UK, in a growing
international context and across an increasingly diverse range of higher education providers.
3
INTRODUCTION
The Quality Assurance Agency for Higher Education (QAA) is a UK-wide agency covering England,
Northern Ireland, Scotland and Wales, within a higher education system that is 'devolved'.
This means that higher education policy is determined by each nation:
l
in England, through Parliament in London
l
in Northern Ireland, through the Northern Ireland Assembly
l
in Scotland, through the Scottish Government
l
in Wales, through the Welsh Government.
UK Parliament
QAA itself operates as a single entity across the whole of the UK. All providers of higher education in
the UK are quality assured via methods aligned to the UK Quality Code for Higher Education
(see page 14), which is published by QAA.
4
UK higher education providers and recognised awarding bodies
The UK has a diverse range of higher education providers, which are independent and autonomous
(not owned by the state).
Higher education providers with the power to award UK degrees are known as 'Recognised Bodies',
and a full list is published by the UK Government's Department for Business, Innovation and Skills.1
There are around 160 providers in the UK that are permitted to award degrees and are recognised
by the UK authorities (UK and Scottish Parliament, Welsh and Northern Ireland Assemblies). The UK
authorities recognise those providers which have been granted degree-awarding powers, either by a
Royal Charter, an Act of Parliament or by the Privy Council (a formal body of advisers to the Queen).
All UK universities and some higher education colleges are Recognised Bodies.
In addition to providers awarding degrees, there are also over 700 colleges and other providers
which do not have their own degree-awarding powers but provide complete courses leading to
recognised UK degrees. These providers are known as 'Listed Bodies'. Courses offered by 'Listed
Bodies' are validated by providers which have degree-awarding powers.
Students and staff
According to the UK's Higher Education Statistics Agency (HESA), UK student and staff numbers for
the 2011-12 academic year were as follows:
Location: England
Scotland
Wales
Northern Ireland
Overall UK
Students* 2,097,230 216,320 131,185 51,905
2,496,645
Staff (academic)152,630
16,735
8,925
3,095
181,385
Staff
(non-academic)159,980
21,720
11,245
3,915
196,860
* Undergraduate and postgraduate.
Of the student numbers provided above, a notable percentage were international students
studying at UK higher education providers. According to HESA, international student numbers
for 2011-12 were:
l
302,680 (12.1%) from non-European Union countries
l
132,550 (5.3%) from other European Union member countries.
Data from HESA on students studying wholly outside the UK with a UK provider in 2011-12 show
76,360 students within the European Union and 494,650 outside the European Union. This includes
both students registered with a UK higher education provider, and also students studying for an
award from a UK provider. The students may be studying on an international campus, by
distance/flexible/distributed learning, or through a collaborative or partnership arrangement.
1
UK Recognised Bodies: www.gov.uk/recognised-uk-degrees
5
Qualifications frameworks
QAA is the 'custodian' of frameworks and reference points for UK higher education qualifications.
They apply to degrees, diplomas, certificates and other academic awards granted by a higher
education provider with degree-awarding powers. The qualifications frameworks are:
•
The framework for higher education qualifications in England, Wales and Northern Ireland (FHEQ)2
(which forms a part of the Credit and Qualifications Framework for Wales)
•
The framework for qualifications of higher education institutions in Scotland (FQHEIS)3
(part of the Scottish Credit and Qualifications Framework).
The Scottish framework, FQHEIS, was verified as compatible with the Framework for Qualifications
of the European Higher Education Area (FQ-EHEA) in December 2006.4 In November 2008, the
FHEQ was verified as compatible with the FQ-EHEA. These verifications represented an important
landmark in the UK's journey to implement the goals of the Bologna Process.5
See the table on page 7 for examples of the typical higher education qualifications at each level of
the FHEQ and the corresponding cycle of the FQ-EHEA. Within each level, the various qualifications
involve different volumes of learning and hence differences in the range of intended learning
outcomes.
In addition, to enable student mobility and to provide information for employers, QAA has worked
with a number of partner organisations to publish Qualifications can cross boundaries: a rough guide
to comparing qualifications in the UK and Ireland.6
QAA (2008) The framework for higher education qualifications in England, Wales and Northern Ireland:
www.qaa.ac.uk/publications/informationandguidance/pages/the-framework-for-higher-education-qualifications-inEngland-Wales-and-Northern-Ireland.aspx
2
QAA (2001) The framework for qualifications of higher education institutions in Scotland:
www.qaa.ac.uk/publications/informationandguidance/pages/fheq-Scotland.aspx
3
4
http://enic-naric.net/documents/qf-Scotland_en.pdf
5
QAA (2011) The Bologna Process in higher education:
www.qaa.ac.uk/publications/informationandguidance/pages/bologna-process-in-he.aspx
6
6
QAA (2011) Qualifications can cross boundaries: A rough guide to comparing qualifications in the UK and Ireland:
www.qaa.ac.uk/publications/informationandguidance/pages/qualifications-can-cross-boundaries.aspx
Examples of the typical higher education qualifications at each level of the
FHEQ and the corresponding cycle of the FQ-EHEA
Typical higher education
qualifications within each level
Doctoral degrees (eg PhD/DPhil
(including new-route PhD), EdD,
DBA, DClinPsy)**
Master's degrees (eg MPhil, MLitt,
MRes, MA, MSc)
Integrated master's degrees***
(eg MEng, MChem, MPhys, MPharm)
Postgraduate diplomas
Postgraduate Certificate in Education
(PGCE)****
Postgraduate certificates
Bachelor's degrees with honours
(eg BA/BSc Hons)
Bachelor's degrees
Professional Graduate Certificate in
Education (PGCE)****
Graduate diplomas
Graduate certificates
Foundation Degrees (eg FdA, FdSc)
Diplomas of Higher Education (DipHE)
Higher National Diplomas (HND)
Higher National Certificates (HNC)*****
Certificates of Higher Education
(CertHE)
FHEQ level*
8
Corresponding
FQ-EHEA cycle
Third cycle
(end of cycle)
qualifications
Second cycle
(end of cycle)
qualifications
7
First cycle
(end of cycle)
qualifications
6
5
Short cycle (within or
linked to the first cycle)
qualifications
4
Notes:
*
F ormerly, the levels were identified as Certificate (C), Intermediate (I), Honours (H), Master's (M) and
Doctoral (D) level.
** Professional doctorate programmes include some taught elements in addition to the research dissertation.
Practice varies but typically professional doctorates include postgraduate study equivalent to a minimum of
three full-time calendar years, with level 7 study representing no more than one third of this.
*** Integrated master's degree programmes typically include study equivalent to at least four full-time academic
years, of which study equivalent to at least one full-time academic year is at level 7. Thus, study at bachelor's
level is integrated with study at master’s level and the programmes are designed to meet the level 6 and level 7
qualification descriptors in full.
**** In April 2005, the Universities Council for the Education of Teachers, the Standing Conference of Principals,
Universities UK and QAA issued a joint statement on the PGCE qualification title. The full statement may be
accessed at www.qaa.ac.uk/assuringstandardsandquality/qualifications.
***** Higher National Certificates (HNCs) are positioned at level 4, to reflect typical practice.
7
UK degrees
Each UK degree must be awarded by a legally approved degree-awarding body (known as a
'Recognised Body') that has overall responsibility for the academic standards and quality of the
qualification. This applies even if all or part of the provision is delegated to another provider.
It is, therefore, important to have a strong process in place to ensure that degree-awarding powers
and the right to be called 'university' (university title) are only granted to higher education providers
which properly merit the powers they seek.
In the UK, the Privy Council is a formal body of advisers to the Queen and one of its responsibilities
is the granting of degree-awarding powers and university title. QAA advises the Privy Council on
applications for degree-awarding powers and university title. All applications are rigorously scrutinised
against guidance and criteria.7 There are different sets of guidance and criteria for the three types of
degree-awarding powers (listed below), which also vary across the four nations of the UK.
There are three types of degree-awarding powers:
i)Foundation Degree awarding powers (FDAP)
Foundation Degree awarding powers give further education colleges in England and Wales
the right to award Foundation Degrees at level 5 of The framework for higher education
qualifications in England, Wales and Northern Ireland.
ii)
Taught degree-awarding powers (TDAP)
Taught degree-awarding powers give higher education providers the right to award bachelor's
degrees and other taught higher education qualifications up to level 7 of The framework for
higher education qualifications in England, Wales and Northern Ireland, and to level 11 of the
Scottish Credit and Qualifications Framework.
iii) R
esearch degree-awarding powers (RDAP)
Research degree-awarding powers give UK higher education providers with TDAP the right
to award doctoral degrees and master's degrees, where the research component (including a
requirement to produce original work) is larger than the taught component when measured
by student effort. These are higher education qualifications up to level 8 of The framework for
higher education qualifications in England, Wales and Northern Ireland, and to level 12 of the
Scottish Credit and Qualifications Framework.
Different criteria apply for TDAP, RDAP and university title applications in Scotland and Northern
Ireland than for applications in England and Wales.
Guidance and criteria for applicants: www.qaa.ac.uk/aboutus/dap/guidance/pages/guidance-criteria.aspx
7
8
Recent UK higher education policy developments
England
In 2011, the UK Government published a White Paper, Students at the Heart of the System,8
which proposed a number of significant higher education policy changes. One of the proposed
changes was to remove the regulatory barriers that prevent equal competition between higher
education providers of all types - including further education colleges and other alternative
providers - to further improve student choice and to support a more diverse sector.
Another proposed change was the introduction of a more risk-based approach to quality assurance
in England (and potentially Northern Ireland). The White Paper proposed that, in future, the nature,
frequency and intensity of external quality assurance would be guided by each higher education
provider's record in quality assurance and the nature of its provision.
During the summer of 2012, the Higher Education Funding Council for England (HEFCE)9 undertook
a consultation with the higher education sector on the more risk-based approach. The results of the
consultation were published in October 2012, in a report entitled: A risk-based approach to quality
assurance: outcomes of consultation and next steps.10
In November 2012, HEFCE issued a letter of guidance11 to QAA on the development of the new
review method. Its key features will include a six-year review cycle for those higher education
providers with a longer track record of successfully assuring quality and standards, and a four-year
review cycle for those providers with a shorter track record. Following further consultation
(opened on 28 January 2013)12 with the sector on the proposed new review method - to be
named Higher Education Review - QAA will adopt this approach in England from the 2013-14
academic year onward. It has also been agreed that Higher Education Review will be adopted in
Northern Ireland in the same timescale.
Northern Ireland
In April 2012, Northern Ireland launched its first higher education strategy, Graduating to Success,13
setting out the direction for higher education policy in Northern Ireland between 2012 and 2020.
Of particular relevance to QAA is an intention to establish a single quality assurance framework for
all higher education provision in Northern Ireland by 2016. The framework will be developed in
consultation with QAA.
Department for Business, Innovation and Skills (BIS) (2011) Students at the Heart of the System: http://bis.gov.uk/
assets/biscore/higher-education/docs/h/11-944-higher-education-students-at-heart-of-system.pdf
8
9
UK higher education funding councils are responsible for public funding of higher education providers.
10
HEFCE (2012) A risk-based approach to quality assurance: outcomes of consultation and next steps: www.hefce.ac.uk/
media/hefce/content/pubs/2012/201227/risk-based%20quality%20assurance%20consultation%20outcomes.pdf
11
HEFCE (2012) Letter of guidance to QAA: www.hefce.ac.uk/pubs/year/2012/201227
12
QAA consultation on Higher Education Review: www.qaa.ac.uk/newsroom/consultations/pages/higher-educationreview.aspx
13
Department for Employment and Learning (DELNI) (2012) Graduating to Success: www.delni.gov.uk/index/
consultation-zone/archived-consultations/archived-consultations-2011/hestrategy.htm
9
Scotland
Scotland finished a review of its quality framework at the end of 2011. It has retained its national
programme of enhancement-led review, comprising a four-year review cycle and an additional
annual report on each provider.
Enhancement-led Institutional Review (ELIR)14 is QAA's review method for universities and other
higher education institutions in Scotland. Its main focus is to consider an institution's strategic
approach to enhancement, placing a particular emphasis on the arrangements for improving
the student learning experience. It also examines the institution's ability to secure the academic
standards of its awards.
Following the 2011 review, QAA Scotland will now provide an additional range of annual briefings
to the Scottish Funding Council, both on each individual higher education provider and also on
the overall health of quality assurance arrangements across the Scottish higher education sector.
The review of the quality framework also retained a model of national Enhancement Themes,15
managed by QAA Scotland.
The third cycle of ELIR is now underway in Scotland and will run from 2012-16. The ELIR method
includes the use of international reviewers on its review teams.
The Scottish Government has also proposed a wide range of reforms for post-16 education;16
although many of these relate only to the college sector, there are specific proposals about
governance, access and funding for the university sector.
Wales
In September 2012, QAA introduced a revised approach to Institutional Review in Wales, including
changes to outcome judgements.17
Also in 2012, the Welsh Government introduced major changes to its funding system, with a new
tuition fee regime for students. The Welsh Government also carried out a consultation on its draft
Further and Higher Education (Wales) Bill,18 which it expects to bring forward in 2013.
The proposals include:
•
arrangements for quality assurance and enhancement of provision
•
provision for effectively funded students' unions and purposeful student charters
•
powers of direct funding for higher education providers
•amendments to the functions of the Higher Education Funding Council for Wales (HEFCW)
as a result of the new funding and student-support arrangements.
QAA Enhancement-led Institutional Review: www.qaa.ac.uk/institutionreports/types-of-review/pages/elir.aspx
14
10
15
Enhancement Themes: www.enhancementthemes.ac.uk
16
Scotland post-16 education reform: www.scotland.gov.uk/topics/education/post16reform
17
QAA Institutional Review (Wales): www.qaa.ac.uk/institutionreports/types-of-review/pages/institutional-review.aspx
18
Welsh Government (2012) Draft Further and Higher Education (Wales) Bill: http://wales.gov.uk/consultations/education/
feandhebill/?lang=en
ABOUT QAA
Origins
In 1997, QAA was established as a single quality assurance service for providers of higher education
in the UK (see also Additional reference materials: Letter from Secretary of State for Education
and Employability, 13 March 1997). QAA brought together the Higher Education Quality Council
(HEQC) and the quality assessment divisions of the Higher Education Funding Councils for England
and for Wales. The Scottish Higher Education Funding Council agreed to contract its quality
assurance activities to QAA soon afterwards.
Mission
The mission of QAA is to safeguard standards and improve the quality of UK higher education.
Corporate legal status
QAA is an independent body, a registered charity and a company limited by guarantee.
Its Memorandum of Association and Articles of Association19 are published on its public website
(see also Additional reference materials: Companies House Certificate of Incorporation).
Financial arrangements
QAA is funded through a number of channels:
•subscriptions from higher education providers (all publicly funded higher education providers
in the UK subscribe to QAA and pay an annual fee, as do some that are not publicly funded)
•contracts and agreements20 with the UK funding councils and organisations to which QAA
reports annually:
­
- Higher Education Funding Council for England
­
- Scottish Further and Higher Education Funding Council, and Universities Scotland
­
- Higher Education Funding Council for Wales and Higher Education Wales
­
- Department for Employment and Learning in Northern Ireland
•providers of higher education seeking educational oversight for immigration purposes
(as required by the UK Border Agency) pay a fee to be reviewed by a QAA team, as well an
annual maintenance charge
•contracts with the General Osteopathic Council (GOsC) and the Teaching Agency
for Early Years Professional Status (EYPS)
•additional private contracts, consultancy and business development work in the UK
and internationally.
QAA Memorandum of Association and Articles of Association: www.qaa.ac.uk/aboutus/corporate/pages/default.aspx
19
20
QAA contracts and agreements: www.qaa.ac.uk/aboutus/corporate/pages/contracts.aspx
11
Governance
QAA Board
QAA is governed by its Board, chaired by Sir Rodney Brooke CBE. The QAA Board21 is responsible for
policy development, for the conduct of the Agency's finances and for monitoring its performance
against agreed targets at a corporate level.
The 17 QAA Board members represent a wide range of interests, both within higher education
and in other areas. The eight independent members of the QAA Board form its largest single group.
A number of Board members have been appointed on the basis of their experience of industry,
commerce, finance or the practice of a profession, and there are also now two student Board
members. In addition, the Board membership includes representatives of UK higher education
and further education providers, and the higher education funding councils.
The QAA Board has nine committees:
•
Access Recognition and Licensing Committee
•
Advisory Committee on Degree-Awarding Powers
•
Audit Committee
•
Business Development Committee
•
Finance and Strategy Committee
•
Nominations Committee
•
QAA Wales Committee
•
QAA Scotland Committee
•
Remuneration Committee.
The membership and terms of reference of all Board committees are set out in the Code of best
practice for members of the QAA Board.22
Chief Executive
The Chief Executive of QAA is Anthony McClaran. The Chief Executive is appointed by and is
accountable to the Board. His role is to provide day-to-day leadership of QAA and its programmes
of work, including the setting and achievement of corporate objectives in line with the Board's
strategic direction. Much of this work involves liaison and consultation with external partners and
stakeholders (for example, Government departments; civil servants; professional, statutory and
regulatory bodies; funding bodies; institutions and their representative bodies; students and their
representative bodies).
The Chief Executive is responsible for:
•the overall organisation, management and staffing of QAA, including its corporate, financial,
estate and personnel management
•the propriety and regularity of QAA's finances, and the efficient, effective and economical use
of resources.
In addition, QAA has five Directors, each responsible for one of the five main organisational groups
(see Organisational structure, page 13).
QAA Board: www.qaa.ac.uk/aboutus/corporate/board/pages/default.aspx
21
22
12
QAA (2011) Code of best practice for members of the QAA Board: www.qaa.ac.uk/publications/informationandguidance/
pages/code-best-practice-qaa-board.aspx
Organisational structure
QAA has five main organisational groups:
•
Public Engagement (Director - Richard Jarman)
•
QAA Scotland (Director - Dr Bill Harvey)
•
Research, Development and Partnerships (Director - Dr Jayne Mitchell)
•
Resources (Director - Douglas Blackstock)
•
Reviews (Director - Dr Stephen Jackson)
QAA has a staff of around 170 employees and a database of approximately 600 reviewers.
Head of
Corporate Affairs
Chief Executive
Director of Public
Engagement
Director of
Reviews
Multimedia
Manager
Deputy Director
Public Relations
Manager
Operational
Support Manager
Student
Engagement
Manager
Head of DegreeAwarding Powers/
University Title
Director of
Research,
Development and
Partnerships
Director of
QAA Scotland
Head of Reviews
Head of Research,
Information and
Enquiries
Head of
Enhancement
Head of Standards,
Quality and
Enhancement
Head of Network
and Partnerships
Director of
Resources
Head of
Information
Management and
Infrastructure
Head of Finance
and Planning
Head of Human
Resources and
Organisational
Development
Head of Access
Head of Wales
and Concerns
Head of Business
Strategy and
Development
Head of
Educational
Oversight
13
QAA'S MAIN AREAS OF WORK IN THE UK
Reviews of higher education providers
QAA conducts evidence-based peer reviews of higher education providers and publishes reports
detailing the findings, to provide public assurance on academic standards, quality and the provision
of public information. QAA carries out reviews using a variety of methods depending on the nation
or the type of higher education provider. All QAA review methods for publicly funded higher
education now have review teams that include a student member.
Separate review methods are used for:
•
reviews of higher education providers
•
reviews of higher education delivered in further education colleges
•
educational oversight reviews for independent providers
•
reviews of collaborative arrangements
•
reviews of professional programmes
•
reviews of international provision.
UK Quality Code for Higher Education
QAA publishes a range of definitive reference points and guidance to support higher education
providers in setting and maintaining academic standards, assuring quality and promoting quality
enhancement. These publications are widely used by UK academic and professional staff responsible
for shaping student learning opportunities. The central reference point is the UK Quality Code for
Higher Education.23
In development since 2008, the Quality Code replaces the previous set of UK national reference
points, known as the Academic Infrastructure, and has been introduced from the
2012-13 academic year.
The Quality Code sets out the expectations that all UK higher education providers are required to
meet. QAA works closely with the UK higher education sector to develop, maintain and update the
Quality Code. The Quality Code is set out in three Parts:
•
Part A: Setting and maintaining threshold academic standards
•
Part B: Assuring and enhancing academic quality
•
Part C: Information about higher education provision.
The Quality Code underlines QAA's belief that all providers of higher education in the UK should be
quality assured under a common framework which can be adapted in its application in different UK
countries, and which also recognises the value of enhancement.
Higher education providers use the Quality Code to help them to set and maintain the academic
standards of their programmes and awards, to assure and enhance the quality of the learning
opportunities they make available, and to provide information about higher education.
Student representatives and students' unions can use the Quality Code in their discussions with their
higher education provider, as it sets out the minimum expectations for the quality of the learning
opportunities the provider makes available to its students. Reviewers carrying out QAA reviews use
the Quality Code as a benchmark for judging whether an individual higher education provider meets
national expectations for academic standards and the quality of learning opportunities.
23
14
UK Quality Code for Higher Education: www.qaa.ac.uk/qualitycode
Feature of good practice
The process for developing the UK Quality Code for Higher Education
In 2009-10, an evaluation and consultation took place of the UK’s Academic Infrastructure - a set of
UK-wide reference points for assuring standards and quality in higher education. This resulted in a
decision to restructure the content into a UK Quality Code for Higher Education in three Parts
(A on standards; B on quality; C on information), subdivided into Chapters, some of which would
be entirely new.
QAA created a detailed protocol24 for the development of the Quality Code. The protocol is published
on the QAA website and a Quality Code Steering Group was formed to oversee its development.
The Quality Code was formally launched in December 2011, and work has continued in 2012
and 2013 to develop the new Chapters and to revise the material taken from the Academic
Infrastructure. The Quality Code will be completed by the end of 2013.
The development of each Chapter or Part of the Quality Code is supported by an advisory
group of practitioners and students who are experts on that particular subject, plus one
representative with expertise in European and international developments in higher education.
The work of QAA and the advisory groups in developing or revising each Chapter or Part of the
Quality Code is supported by a public consultation with the higher education sector and other
stakeholders with an interest in higher education.
The process for developing and/or revising a Chapter or Part of the Quality Code takes,
on average, around one year to complete.
Enhancement
QAA supports higher education providers in enhancing the quality of the education they provide,
through the sharing of good practice and ideas. QAA offers advice and guidance to help UK higher
education providers deliver the best possible student experience.
Where there are areas of common interest, QAA works in partnership with the UK's Higher
Education Academy,25 which champions excellent learning and teaching in higher education.
QAA review reports identify features of good practice and QAA publishes overview reports on good
practice findings in its Outcomes26 series. These are useful resources for quality managers and those
who design academic programmes of study.
QAA also conducts research that is helpful to those responsible for quality. In 2012, a QAA research
strategy was approved to provide a focus and framework for QAA to undertake research in support
of the QAA Strategy 2011-14 and annual priorities.
QAA also seeks to cultivate debate and collaboration that supports improvement, hosting events
and publishing discussion papers such as the Talking about quality27 series, inviting feedback from all
those interested in the quality of UK higher education - from students and parents to policymakers
and senior academics.
P
rotocol for developing the UK Quality Code: www.qaa.ac.uk/publications/informationandguidance/pages/protocol-fordeveloping-the-quality-code.aspx
24
Higher Education Academy: www.heacademy.ac.uk
25
QAA Outcomes series: www.qaa.ac.uk/improvinghighereducation/pages/reviews.aspx
26
27
QAA Talking about quality series: www.qaa.ac.uk/improvinghighereducation/debate/pages/default.aspx
15
In Scotland, enhancement has been fully integrated into the approach taken to quality assurance,
through an enhancement-led review methodology and developmental framework. The strength of
the enhancement-led approach has been recognised and endorsed by the Scottish Government in
its most recent White Paper,28 Putting Learners at the Centre:
[The] higher education quality enhancement framework [is] based on three principles of high
quality learning, learner engagement and a quality culture. The distinctive Scottish approach,
particularly in relation to universities, is recognised internationally as being successful in
sustaining and promoting quality (paragraph 182).
Advice to Government on degree-awarding powers and university title
(See also 'UK degrees', page 8.) QAA's Advisory Committee on Degree-Awarding Powers (ACDAP)
considers applications initially and decides if they should proceed to detailed scrutiny. If ACDAP
recommends that an application should proceed, QAA appoints a team to carry out a detailed
scrutiny of the application and supporting evidence.
Applications are considered against the relevant set of criteria determined by the four nations of
the UK (England, Northern Ireland, Scotland, Wales) and are designed to establish that an applicant
is a well-founded and self-critical academic community that can demonstrate firm guardianship of
its standards. QAA's advice is also sought on applications for university title - the right to be called
a university.
Working with students
QAA works with students as partners in quality assurance, involving them in QAA governance.
There are two student members of the QAA Board, plus 17 members of the QAA Student Advisory
Board. There are also over 80 trained student reviewers now available to take part in QAA reviews.
QAA works with students to develop QAA policy and the Quality Code. There are student members
of the QAA Scotland Committee and QAA Wales Committee, and of the committees which manage
and oversee the work of the Enhancement Themes.
Investigating concerns about higher education providers
QAA investigates concerns about academic quality and standards where there is evidence of
systemic and/or procedural weaknesses. Concerns can be raised with QAA by students, staff within
higher education providers or other interested parties, either from within the UK or internationally.
Regulation of the Access to Higher Education Diploma
Access to Higher Education29 (Access to HE) courses are designed for people who want to take a
university-level course, but who did not gain the qualifications they needed while at school.
Courses are available at colleges across England and Wales, in a wide range of subjects.
Since its founding in 1997, QAA has been responsible for assuring the quality of recognised Access
to HE courses. To meet these responsibilities, QAA has developed the Recognition Scheme for
Access to Higher Education. QAA also licenses regionally based Access Validating Agencies (AVAs),
authorising them to recognise Access to HE programmes and to award the Access Diploma to
students. In addition, QAA provides advice to the UK Government on the Access to HE programme,
as required.
28
Scottish Government White Paper: www.scotland.gov.uk/publications/2011/09/15103949/15
Access to Higher Education: www.accesstohe.ac.uk
29
16
Commercial activity
QAA Enterprises is the commercial arm of QAA, specialising in quality improvement in higher and
further education. QAA Enterprises delivers training, events and enhancement services to enable
higher education providers and other stakeholders to benchmark, improve and manage their own
quality assurance processes.
International activity
QAA and the UK higher education sector continue to benefit from engagement with European and
wider international quality assurance developments. QAA takes a proactive role in international
developments in standards and quality, driven by its international strategy, which has three
main goals:
•to secure academic standards in internationally delivered UK higher education or
transnational education (TNE)
•to be an authority on international quality assurance and enhancement matters, increasing
recognition of QAA as a provider of authoritative advice by sharing expertise, intelligence
and information
•to maximise the influence of QAA in international quality assurance developments.
Feature of good practice
Reviews of overseas provision/transnational education
Many UK degree-awarding bodies offer some of their higher education programmes through
partnership links with organisations outside the UK. The UK degree-awarding bodies are responsible
for the academic standards of their awards, whether delivered within or outside the UK.
QAA reviews the partnership arrangements between UK degree-awarding bodies and organisations
in other countries to check that they are robust and that UK expectations are being met.
In late 2012, QAA undertook a transnational education (TNE) review in mainland China. This review
was a pilot for a new QAA review method for TNE, on which QAA plans to publish a new review
handbook in 2013.
Detailed information on the methods used in QAA reviews of overseas provision and
transnational education can be found in the review reports.30
QAA was a founder member of the European Association for Quality Assurance in Higher Education
(ENQA). Through its ENQA membership, QAA demonstrates the compatibility of quality assurance
arrangements in the UK with the Standards and Guidelines for Quality Assurance in the European
Higher Education Area.
QAA is a member of the International Network for Quality Assurance Agencies in Higher Education
(INQAAHE) and participates in the Asia-Pacific Quality Network (APQN). QAA is also a member of
the Council for Higher Education Accreditation (CHEA) International Quality Group, and participates
in many forums and dialogues with international partners. In addition, QAA has links with partner
agencies around the world through Memoranda of Understanding and Cooperation.
The Chief Executive of QAA is a member of the International Education Advisory Forum (IEAF),
chaired by the UK's Minister of State for Universities and Science, and QAA is a stakeholder in
the UK India Education and Research Initiative, and UK China Partners in Education
cooperation programmes.
QAA Review of Overseas Provision: www.qaa.ac.uk/institutionreports/types-of-review/overseas/pages/default.aspx
30
17
QAA has close relationships with international quality assurance agencies, monitoring and reporting
on advances around the world. QAA publishes a monthly newsletter, Quality Update International,31
which covers news and a selection of articles relating to higher education and quality assurance.
UK higher education sector partnerships and leadership
QAA consults and works with many different stakeholder groups with an interest in the quality of UK
higher education. These include:
•
students and the UK's National Union of Students
•
government bodies
•
higher education funding bodies/councils
•
other higher education sector agencies (such as the Higher Education Academy)
•
higher education staff
•employers
•
professional, statutory and regulatory bodies
•
representative bodies for higher education.
QAA also takes a leadership role within the higher education sector, working with departments of
government across the UK's four nations and other sector bodies. It is a member of a number of key
leadership groups in the UK, as follows.
Higher Education Better Regulation Group (HEBRG)
HEBRG was established in 2010 to address changing public and political attitudes towards higher
education accountability, and the evolving regulatory landscape. HEBRG members include higher
education representative and funding bodies, and regulators from the four UK nations.
Professional, Statutory and Regulatory Bodies (PSRB) Forum
Established in June 2008, the PSRB Forum is a joint venture between QAA and the UK
Inter-Professional Group (UKIPG). The PSRB Forum enables representatives from UK PSRBs to
share ideas and experiences, to discuss areas of mutual interest, and to share good practice.
Quality in Higher Education Group (QHEG)
QAA is a member of QHEG, which aims to promote a quality assurance system for higher education
in England and Northern Ireland that is accountable, rigorous, transparent, flexible, responsive,
enhancement led and public facing. Its members are drawn from sector agencies, universities,
students, further education and the Royal Academy of Engineering.
Regulatory Partnership Group (RPG)
The RPG was established in September 2011 to advise the UK Government, HEFCE and other
national agencies on policy, strategic and operational issues arising from the development of the
new funding and regulatory regime for higher education in England.
Research Advisory Board
Established in 2012, the Research Advisory Board advises on the implementation of QAA's research
strategy and the development of capacity for applied research in higher education, and to promote
awareness of QAA's role in higher education research. The Research Advisory Board has nine
external members from higher education providers and the National Union of Students.
Quality Update International: www.qaa.ac.uk/international/pages/quality-update-international.aspx
31
18
Scottish Credit and Qualifications Framework (SCQF) Partnership
QAA was a founding member of the SCQF Partnership, which was established in November 2006
to oversee the quality and promotion of the SCQF, and maintain links with frameworks in the rest of
the UK and internationally.
Scottish Higher Education Enhancement Committee (SHEEC)
Through support for - and promotion of - quality enhancement, SHEEC ensures that higher education
in Scotland remains at the forefront of developing and enhancing the student learning experience.
Student Experience, Teaching and Quality Committee (SETQC)
SETQC advises the Higher Education Funding Council for Wales (HEFCW) on strategic policy
issues relating to learning and teaching, and undertakes HEFCW's statutory responsibility for
quality assessment.
Student participation in quality Scotland (sparqs)
QAA is a member of the steering committee for sparqs, which delivers support services to
student representatives throughout the Scottish higher education sector.
Universities Quality Working Group
QAA Scotland is a member of the Universities Quality Working Group, which coordinates the
Scottish Quality Enhancement Framework, supporting higher education providers in managing
the quality of the student learning experience, and providing public confidence in the quality and
standards of higher education.
19
QAA STRATEGY 2011-14
QAA's vision is to be the authority on UK higher education standards and quality. The Agency's
mission is to safeguard standards and improve the quality of UK higher education.
QAA has four strategic aims that will help it to pursue its vision and achieve its mission.
AIM ONE: to meet students' needs and be valued by them
Aim
and 1: To
be v meet
We
alu
w
ed b studen
edu ill work
ca
ts
y th
as th tional to ens
u
exp
e
em ' nee
erie re that
com y aim
We will work to ensure that all students get the best possible educational
experience. We will support universities and colleges as they aim to
meet and shape students' expectations. We will communicate clearly
to students about standards and quality, and will work with them as
partners. We will respond to the views and diverse needs of students,
and will protect their interests.
ds
nce
all
to m
mu
stu
. W
n
eet
wil
d
ew
l wo icate c
a
ill su ents g
learl nd sh
rk w
and
et
ape
ppo
y to
ith
div
rt u the be
stu
the
ers
de
st p
niv
m a studen
e ne
ers
s pa
ts a ntsÊ ex
eds
itie ossible
rtn
bou
p
of s
s an
How
ers
t sta ectati
tud
d co
.W
ons
ents
nda
wil
ew
ll
.W
rds
and
ill re
l we
e w eges
a
w
n
ill
s
d
ill p
p
Sur
kno
rote ond to quality
vey
w
we
s
ct th
the
,
we
pro confi
view and
eir
vid
rmin
a
inte
re s
s
e
g th
res
Gro
ucc
ts.
at s
w
eed
tud
can th in
ents
ing
e
infl
valu
uen nquirie
?
e th
ce
s fr
Res
e be
om
earc
n
s
efits
tud
h sh
res
ents
ult
on
of o owing
issu
ur w
s
es w
ork pecific
e
wit
h stu impro
vem
den
e
nts
ts
to p
rac
tice
as a
AIM TWO: to safeguard standards in an increasingly
diverse UK and international context
As an independent, trusted authority, we safeguard standards,
enabling students, employers and the wider public to have confidence
that UK higher education qualifications have a recognised value. We
will take an energetic approach to new challenges, adapting quality
assurance systems in the UK where necessary to ensure they remain
effective and proportionate, targeted on risk and need in an increasingly
diverse sector. We will protect the interests of everyone working towards
a UK higher education qualification, however and wherever they study,
within the UK or abroad.
s
andard
uard st e UK
o safeg
rs
T
e
:
iv
2
gly d
Aim
creasin l context
a
in an in
ill we
rnation
How w
we are
and inte
at
d
nt, truste
know th ng?
ds,
epende
standar
di
As an ind
feguard
succee
y, we sa
ployers
that
authorit
nts, em
showing
ve
g stude
Surveys ds across the
c to ha
enablin
bli
on
pu
ati
e
tho
wider
our me
effectiv
her educ
and the
seen as
t UK hig
nised
UK are
nce tha
ble
a recog
lua
confide
ve
va
ha
c
and
ations
energeti
rs
be
an
qualific
e
cri
tak
subs
her
We will
enges,
More the whole hig
value.
w chall
stems
, and
across
ch to ne
rance sy
on sector
approa
ty assu
tended
educati
g quali
remain
iews ex
adaptin
re they
QAA rev e range of
to ensu
te. We
ers
div
a
in the UK d proportiona
to
l
rs
e an
nationa
provide
effectiv
differing
target
t our
pond to
need to
nce tha
will res
used
as the
ote
Evide
s are foc
d prom
es such
st
resource they add mo
prioriti
ersity an
ere
re
pport div
otect the
wh
su
pr
on
ll
k,
ere the
wi
ris
and wh
s
ent. We
ing
lue
em
rk
va
nd
nc
wo
sta ard
enha
one
risks to
on
of every
are
ati
sts
uc
intere
ality
her ed
erever
and qu
a UK hig
and wh
towards
wever
ad.
or abro
ation, ho
qualific
n the UK
dy, withi
they stu
AIM THREE: to drive improvements in UK higher education
Our review reports explain our findings clearly and authoritatively.
We identify what is good and where improvements are needed, and
communicate this information widely to promote improvements
in students' experiences. We maintain and develop our role as
independent experts on standards, quality and enhancement,
valued across the higher education sector. We build on our
reputation, offering our services more widely, sharing our expertise
publicly and encouraging debate and positive change.
We investigate complaints quickly, and expect universities and
colleges to act on our findings.
AIM FOUR: to improve public understanding
of higher education standards and quality
We will describe and discuss standards and quality in higher
education in ways that the public understand. Our publications will
be accessible and useful to a diverse audience, including students,
prospective students and interested members of the public. In judging
the information universities and colleges provide, our work will
encourage improvement in this area. We will strengthen our research
base, lead policy developments and contribute to public debate.
20
Aim
in U 3: To
dr
Kh
Our
igh ive im
re
er e
auth view
re
o
duc provem
imp ritativ ports
rov
wil
e
atio
l ex
eme ly. W
wid
ent
e
p
nts
e
n
are will id lain ou
s
wil ly to p
enti
ne
ro
lm
r fin
fy
din
stan ainta mote im eded,
gs c
and what
in a
d
pro
is
lea
nd
com
edu ards,
vem
de
qu
c
mu good a rly an
ents
our ation se ality a velop
in st nicate nd wh d
ou
serv
n
c
ere
this
ude
enc
ices tor. W d enha r role
n
info
ou
ts
a
e
nce
rm
com ragin more w will b
men s indep Ê expe
u
rien ation
g
p
end
t, va
ide
ild
ces.
ly,
on
our laints debate
our lued a ent ex
quic
find
We
and shari
p
cro
re
n
kly
ing
ss th erts on
s.
and positiv g our e putati
e hig
on,
e ch
xpe
exp
How
off
rt
h
ect
a
univ nge. W ise pu ering er
wil
blic
e
e
rs
l
w
we
itie
ly a
s an ill inve
Goo
nd
kno
st
d co
dp
ww
ra
lleg igate
our
es to
web ctice e
e ar
vid
site
act
e su
enc
on
Evid
ed in
cce
enc
a
e
ll in
din
upo
e th
stit
nq
g?
utio
uic at our
kly
ns a
find
and
Inc
n
in
d fe
gs a
rea
ma
atu
se in
nd
ke a
red
our
diffe recom
on
spe
men
ren
ed o
ce
dati
f re
ons
spo
are
nse
a
cted
to c
onc
ern
s ra
ised
blic
rove pu r
e
To imp
Aim 4: nding of high
uality
ta
s and q
unders
andard
higher
st
n
ality in
o
and qu
cations
r publi
ndards
educati
Ou
cuss sta
rstand.
luding
e and dis public unde
nce, inc
describ
the
of the
se audie
We will
ys that
mbers
a diver
ted me
on in wa and useful to
lleges
interes
educati
d
ble
an
and co
ssi
dents
acce
rsities
ea. We
will be
this ar
ctive stu mation unive
pe
in
t
os
and
, pr
or
emen
lopments
students
g the inf urage improv
ve
gin
de
licy
In jud
enco
, lead po
public.
rk will
ch base
, our wo
r resear
provide
ou
en
ength
bate.
will str
blic de
?
te to pu
eeding
contribu
are succ
ess of
ow we
awaren
public
ill we kn
reased
How w
ing inc
ch show on
ar
e
se
ucati
ucts ar
rket re
er prod
Ma ty in higher ed
d
oth
d
accesse
ns an
quali
quently
blicatio
our pu
and fre
nces to
owing
useful
sh
ere
le,
ref
ab
sts
ad
more
Te
ingly re
te and
increas
c deba
in publi
profile
onger
b
A str rk on the we
our wo
QAA has four commitments upon which its strategy is founded.
The intrinsic worth of higher education
QAA admires and supports the research and teaching that takes place in higher education providers
across the UK. It respects the autonomy of UK providers, and believes that it fosters the diversity
that is central to their success and international reputation. QAA also recognises that the primary
role of higher education providers in maintaining academic standards and quality is vital to that
autonomy. QAA relies upon their cooperation in its work, and in return provides valuable advice
and support.
The entitlements of students
All students deserve a high-quality learning experience. They have a right to a range of learning
opportunities leading to a qualification that has recognised value and meets published national
expectations. Students are partners in quality assurance, and are experts not only in their own
learning but also on issues of governance, policy and practice. QAA seeks to harness that expertise
in every aspect of its work.
The public interest in higher education
Students, their families and the wider public make a big investment in higher education. As well
as helping students meet material aspirations and offering personal fulfilment, higher education
enriches society. QAA believes the public has a legitimate interest in ensuring standards are
safeguarded and quality maintained, and that QAA has a duty to communicate its work clearly to a
wide audience.
The importance of equality and diversity
QAA believes that equality and diversity should be promoted through the services it provides and
that in its work, it should be supportive, fair, just and free from discrimination. The higher education
sector should lead the way in valuing the diverse contributions of all staff, students and partners,
and in developing and sharing good practice in this area.
21
QAA PROGRESS SINCE THE 2008 ENQA REVIEW
In its 2008 report on QAA, the ENQA review team highlighted a number of areas for development.
Below is a chart summarising progress made in key areas since 2008.
2008 ENQA report
recommendation
Inclusion of international
reviewers in QAA review
activity (paragraph 25)
QAA progress
Since its 2008 review by ENQA, QAA has piloted and evaluated the
inclusion of international reviewers on its review teams across a full
cycle of Enhancement-led Institutional Review (ELIR) in Scotland.
International reviewers were introduced into the ELIR review
methodology in 2008 and have been used in each institutional
review of Scottish higher education providers in the 2008-12 cycle.
The international reviewers were able to provide assurance that the
quality systems in place in Scottish institutions were consistent with
expectations in a range of other countries. In addition, they were
able to provide developmental insights in areas of academic practice,
drawing on the role of similar practice in their own professional
context. In many cases, QAA was able to use the expertise and
networks of international reviewers to support its wider enhancement
activities in Scotland.
The pilot has proved to be a very positive experience, bringing added
depth and dynamism to the review teams, with some logistical
considerations (such as travel and visa arrangements) to be further
explored for the future.
In December 2012, the QAA Board reviewed the evaluation of the
pilot in Scotland and endorsed the inclusion of international reviewers
in other QAA review methods in future, where they can add real
value in a meaningful way. This would begin with the new
Higher Education Review method (see page 9).
More accessible public
information for nonspecialist audiences
(paragraph 32)
Since the 2008 ENQA review, QAA has made a dramatic and positive
shift in its approach to public engagement. Key to this was a review
of our communications in 2009-10, which led to the development
of a new approach to public engagement. This approach had four
key aims:
Aim 1: Improve public understanding of UK higher education quality
Aim 2: Reduce the clutter in QAA communications
Aim 3: Be visible and valued in public debate
Aim 4: Manage resources strategically
These aims are now part of business planning and the QAA Strategy
2011-14. Internal teams were restructured to merge responsibility for
review reporting and all other communications into one group, the
Public Engagement Group.
22
Specific differences in reporting since 2008
•Every higher education provider now has its own page on the
QAA website, summarising key information and outcomes of
recent reviews. Readers can access this summary information
without having to download the full report.
•Published QAA reports are now much more accessible, written
in clear English and include a glossary of terms to explain the
more complex information to a non-technical (public) audience.
•QAA releases its review judgements about each higher
education provider to the media in the provider’s local area.
•In August 2012, QAA launched a QAA Quality Mark, which
eligible higher education providers who have undergone a
positive QAA review can display on their websites.
•A new corporate website, www.qaa.ac.uk, was launched in
July 2011, designed to address the needs of public and
student audiences.
Development of a more
detailed protocol detailing
the relationship between
QAA, the funding councils
and their respective
Boards, set within a longer
planning horizon than at
present (paragraph 57)
Inclusion of a student
member on review panels
(paragraph 60)
QAA has service level agreements/contracts with funding councils in
each of the four nations of the UK. The QAA Board has established
a Finance and Strategy Committee, and six-yearly financial forecasts
and income generation targets have been set. This enables QAA to
take a longer-term view when negotiating with the funding bodies,
setting subscription levels and targeting opportunities for
business development.
Feature of good practice
Student engagement
Since the 2008 ENQA review, QAA has made major changes to its
approach to students and student engagement.
QAA has worked to involve students as equal members of review
teams across all of its new review methods, as they come into effect.
There are now over 80 trained student reviewers available to take
part in review activity. QAA also supports students involved in the
review process - in 2011-12, supporting over 200 lead student
representatives in their roles across all its review methods.
QAA is now undertaking development work to see how its student
engagement can be extended and deepened. It has developed
student-specific guidance documents and web resources for all
relevant review methods.
In Wales, QAA is a partner in the Welsh Future Directions programme
looking at quality enhancement themes, and is represented on the
Students as Partners workstream.
23
QAA is also a partner in the Wales Initiative for Student
Engagement (WISE).
In Scotland, student engagement is a central part of the Scottish
Quality Enhancement Framework. QAA Scotland has included
student reviewers as full members of all institutional review teams
since 2003. In order to support student representatives in colleges
and universities, a development body known as sparqs (student
participation in quality Scotland) was created in 2003.
QAA Scotland was a founding member of the sparqs partnership
and is represented on its management committee.
More fully involve QAA
staff and stakeholders
in the development
of the self-evaluation,
reflecting on the
Agency’s performance
and opportunities for
enhancement
(paragraph 68)
As part of the development of its 2013 self-evaluation report,
QAA has undertaken extensive staff and wider stakeholder
engagement (including an analysis of strengths, weaknesses,
opportunities and threats). Engagement activity has included:
•establishment of an internal ENQA review project board and
project team
•
workshop with QAA Board members on 1 October 2012
•two staff workshops at QAA Gloucester offices on
31 October 2012
•survey with six UK professional, statutory and regulatory bodies
in November 2012
•consultation workshop with QAA’s Student Advisory Board on
16 November 2012
•consultation workshop with members of the QAA Sounding
Board (parties that contribute to QAA funding) on
19 November 2012
•
staff workshop at QAA Glasgow offices on 20 November 2012
•discussion at QAA Scotland Committee and QAA Wales
Committee in December 2012.
All of the input, expertise and advice of these stakeholders has been
used to inform and directly influence this self-evaluation report
(see page 57).
24
QAA’S COMPLIANCE WITH EUROPEAN
STANDARDS AND GUIDELINES (PART 2)
Standard 2.1
Use of internal quality assurance procedures
STANDARD
External quality assurance procedures should take into account the effectiveness of
the internal quality assurance processes described in Part 1 of the European Standards
and Guidelines.
Guidelines
The standards for internal quality assurance contained in Part 1 provide a valuable basis for the
external quality assessment process. It is important that the institutions' own internal policies and
procedures are carefully evaluated in the course of external procedures, to determine the extent
to which the standards are being met.
If higher education institutions are to be able to demonstrate the effectiveness of their own
internal quality assurance processes, and if those processes properly assure quality and standards,
then external processes might be less intensive than otherwise.
How QAA meets this standard
QAA review methodologies articulate with the UK Quality Code for Higher Education32 and/or other
agreed reference points. The Quality Code has a comprehensive range of Chapters setting out
agreed good practice in the form of Expectations which higher education providers are expected
to meet. The Expectations are illustrated by Indicators of sound practice that set out ways in which
adherence to the Expectations might be achieved.
The Quality Code, and the corresponding headings under which reviewers are expected to report,
address all of the standards set out in Part 1 of the Standards and Guidelines for Quality Assurance
in the European Higher Education Area (ESG). The table on pages 26-27 shows the relationship
between the ESG and the Quality Code. It is important to note that the Quality Code is developed
in consultation with the higher education community and draws on their good practice in internal
quality assurance. When higher education providers write their self-evaluation documents as part of
the QAA review process, they respond to the Quality Code in order to demonstrate the effectiveness
of their internal procedures.
QAA reviews are moving towards a more risk-based approach, whereby the intensity or frequency
of the external quality assurance process is determined by the provider's record in quality assurance
(see also 'Recent UK higher education policy developments' on page 9 and compliance with
Standard 2.7).
QAA is, therefore, confident that the external quality assurance procedures that it uses take into
account fully the effectiveness of the internal processes described in Part 1 of the ESG.
UK Quality Code for Higher Education: www.qaa.ac.uk/qualitycode
32
25
Relationship between Standards and Guidelines for Quality Assurance in the
European Higher Education Area and the UK Quality Code for Higher Education
ESG for internal
quality assurance
within higher
education
institutions
1.1 Policy and
procedures for
quality assurance
Relevant Indicators of sound
practice in the Quality Code
Quality Code
•
General Introduction
•Part A: Setting and
maintaining threshold
academic standards
•Part B: Assuring and
enhancing academic
quality
•Part C: Information
about higher education
provision
•General Introduction - especially
overarching principles (p 4)
•
B3: Learning and teaching - Indicator 1
•
B9: Complaints and appeals on
academic matters - Indicator 2
•
B10: Managing higher education
provision with others - Indicators 1-3
•
B11: Research degrees - Indicators 1-2
•Part C: Information about higher
education provision - Indicator 7
•Quality enhancement is embedded in
Part B rather than being the subject
of a specific Chapter.
There are, however, examples of
specific Expectations and Indicators
voicing elements of enhancement
such as:
•
B3: Learning and teaching Expectation
•
B5: Student engagement Indicators 1 and 2
•Part C: Information about higher
education provision - Indicator 7
1.2 Approval,
monitoring and
periodic review of
programmes and
awards
•Part A, Chapter A4:
Approval and review
•Part B, Chapter B1:
Programme design and
approval
•Part B, Chapter B8:
Programme monitoring
and review
26
•
10: Managing higher education
B
provision with others - Indicators 13
and 17
1.3 Assessment of
students
•Part A, Chapter A6:
•
B3: Learning and teaching Assessment of achievement
Indicator 9
of learning outcomes
•
B7: External examining is highly
significant for assessment and a
•Part B, Chapter B6:
unique feature of the UK system
Assessment of students
and accreditation of prior
•The right to appeal an assessment
learning
decision is also significant B9: Complaints and appeals on
academic matters (revised version of
Chapter to be published at end of
April 2013)
•
B10: Managing higher education
provision with others - Indicator 15
•
B11: Research degrees - Indicators 13,
16, 17 and 18
1.4 Quality assurance •Part B, Chapter B3:
of teaching staff
Learning and teaching
•Indicator 4 - note in particular
that this Indicator endorses the UK
Professional Standards Framework,
published by the Higher Education
Academy
•
B3: Learning and teaching deliberately
takes a broad interpretation of
'teaching' to include support
for learning
•
B11: Research degrees - Indicators 9
and 11
1.5 Learning
resources and
student support
1.6 Information
systems
1.7 Public
information
•Part B, Chapter B4:
Student support, learning
resources and careers
education, information,
advice and guidance
•
B3: Learning and teaching - Indicator 6
•
Chapter B4 to be published as
B4: Supporting student achievement
on 30 March 2013
•
B11: Research degrees - Indicators 4
and 14
•Part C: Information
about higher education
provision (Indicator 7)
•
B3: Learning and teaching Indicator 5
•Part C: Information
about higher education
provision
•
B5: Student engagement Indicators 5 and 7
•
9: Complaints and appeals on
B
academic matters - Indicator 9
•
11: Research degrees - Indicators 3
B
and 15
•
B3: Learning and teaching Indicator 7
•
B9: Complaints and appeals on
academic matters - Indicator 4
•
B10: Managing higher education
provision with others - Indicator 18
27
Standard 2.2
Development of external quality assurance processes
STANDARD
The aims and objectives of quality assurance processes should be determined before the
processes themselves are developed, by all those responsible (including higher education
institutions) and should be published with a description of the procedures to be used.
Guidelines
In order to ensure clarity of purpose and transparency of procedures, external quality assurance
methods should be designed and developed through a process involving key stakeholders,
including higher education institutions. The procedures that are finally agreed should be
published and should contain explicit statements of the aims and objectives of the processes
as well as a description of the procedures to be used.
As external quality assurance makes demands on the institutions involved, a preliminary impact
assessment should be undertaken to ensure that the procedures to be adopted are appropriate
and do not interfere more than necessary with the normal work of higher education institutions.
How QAA meets this standard
Common principles are followed for the development of all QAA's review methods. However, there
are differences in how the different methods are operationalised. In general, following the receipt
of a commissioning or contract letter from the appropriate body, a draft handbook and - for most
methods - an operational description are developed and consulted on across the higher education
sector. Following the consultation, changes are made to the handbook as appropriate, and it is
published with ample advance time allowed for higher education providers to become familiar
with it and with the method it describes.
This work is reinforced by briefing meetings for institutional representatives and a preparatory
meeting33 at the institution. These are usually carried out by the QAA officer who will be
coordinating the review, in order to ensure that the institution is as well prepared as possible and
has more than one opportunity for clarification.
Commission
the method
Contract or commission
Describe the
method
Consult on
the method
Publish the
method
33
28
Input from
commissioning
authority
Operational description and
handbook developed
Input from
expert advisory
group(s)
Consultation
Operational description
and/or handbook published
E xamples of the agenda for the preparatory meeting are provided in annex 7 of the following handbooks:
www.qaa.ac.uk/publications/informationandguidance/pages/ireni-handbook-second.aspx
www.qaa.ac.uk/publications/informationandguidance/pages/rche-handbook.aspx
Standard 2.3
Criteria for decisions
STANDARD
Any formal decisions made as a result of an external quality assurance activity should be
based on explicit published criteria that are applied consistently.
Guidelines
Formal decisions made by quality assurance agencies have a significant impact on the
institutions and programmes that are judged. In the interests of equity and reliability, decisions
should be based on published criteria and interpreted in a consistent manner.
Conclusions should be based on recorded evidence and agencies should have in place ways of
moderating conclusions, if necessary.
How QAA meets this standard
All QAA review methodologies are developed in consultation with the higher education sector as
described under Standard 2.2. All review method handbooks are available on the QAA website
and contain information on the review process itself, as well as on the judgements and on the
expectations that must be fulfilled in order to achieve positive judgements.
The methodologies set out in the handbooks are reinforced in several ways. Peer reviewers,
including student members of the team, attend a training programme which takes them through
the review method before they are allowed to act as a reviewer. Institutions under review are
invited to attend briefings or preparatory workshops and for the Institutional Review in England
and Northern Ireland (IRENI) and Enhancement-led Institutional Review methods (see 'QAA review
methods at a glance', page 38-39) institutions are also able to access briefing material online.
The expectations against which judgements are made are those set out in the Quality Code and
are mirrored in the handbook for the relevant method, developed in partnership with the higher
education sector. A QAA officer or contracted review coordinator is involved throughout the review
process, and works with the review team on the final day of the review to ensure that judgements
and outcomes of the review are evidence-based and sound. The judgements and outcomes are
subject to scrutiny through an internal moderation process, in order to ensure consistency of
judgements. The report is then drafted or edited by the QAA officer. All such QAA officers have
attended reviewer training and observed/shadowed a more experienced officer, before carrying out
their first review.
Over the last five years, much work has been carried out to ensure that reports and outcomes of
reviews are written in clear and accessible language to meet the needs of a wide range of audiences.
This is an area in which QAA is seeking to learn from other organisations, both in the UK and further
afield, in order to improve the clarity of its reports further. QAA will be interested in the outcomes of
the ENQA project on reporting.
29
Standard 2.4
Processes fit for purpose
STANDARD
All external quality assurance processes should be designed specifically to ensure their
fitness to achieve the aims and objectives set for them.
Guidelines
Quality assurance agencies within the EHEA undertake different external processes for different
purposes and in different ways. It is of the first importance that agencies should operate
procedures which are fit for their own defined and published purposes. Experience has shown,
however, that there are some widely used elements of external review processes which not only
help to ensure their validity, reliability and usefulness, but also provide a basis for the European
dimension to quality assurance.
Amongst these elements the following are particularly noteworthy:
•insistence that the experts undertaking the external quality assurance activity have
appropriate skills and are competent to perform their task
•
the exercise of care in the selection of experts
•
the provision of appropriate briefing or training for experts
•
the use of international experts
•
participation of students
•ensuring that the review procedures used are sufficient to provide adequate evidence to
support the findings and conclusions reached
•the use of the self-evaluation/site visit/draft report/published report/follow-up model
of review
•recognition of the importance of institutional improvement and enhancement policies
as a fundamental element in the assurance of quality.
How QAA meets this standard
The purpose of QAA's external review processes is to provide public assurance about the standards
of higher education awards and the quality of the learning opportunities in place to enable students
to achieve those awards. Processes should not be unduly onerous but should be sufficient to secure
the purpose.
Reviewers ('experts') are nominated by their institution and selected, according to specific criteria,34
through a paper-based exercise. Student reviewers may also be nominated by their students' union.
All selected reviewers must complete a training programme (see feature of good practice, page 31).
34
30
For example, see nomination form for ELIR reviewers:
www.qaa.ac.uk/workwithus/documents/elir%20international%20reviewer%20nomination%20form_2008.doc.
Feature of good practice
Reviewer training
All selected reviewers must complete a training programme which, as far as possible, takes them
through a simulated review which mirrors all the activities undertaken in an actual review. If a
reviewer is unable to complete the training, they will not be allocated to a review. Students are
full members of review teams and are expected to complete the same training as other reviewers.
Efforts are made to train review teams together to assist in team building before the review begins.
QAA’s Single Equality Scheme (see compliance with Standard 3.8) and the selection criteria together
ensure that there is no discrimination.
A new performance management process is being implemented, through which the performance
of reviewers and the QAA officer or contracted review coordinator is evaluated by review team
members, the officer and the provider under review. Feedback is then given to the review team
and the QAA officer who coordinated the review. These evaluations help to confirm that the review
method is fit for purpose and meets its specified aims.
In addition, in late 2012, the first survey of reviewer opinion was conducted by QAA, carried out by
an independent consultant. (See also compliance with Standard 3.8.)
Since its 2008 review by ENQA, QAA has piloted and evaluated the inclusion of international reviewers
on its review teams across a full cycle of Enhancement-led Institutional Reviews (ELIR) in Scotland.
International reviewers were introduced into the ELIR review methodology in 2008 and have been
used in each institutional review of Scottish higher education providers in the 2008-12 cycle.
The international reviewers were able to provide assurance that the quality systems in place in Scottish
institutions were consistent with expectations in a range of other countries. In addition, they were
able to provide developmental insights in areas of academic practice, drawing on the role of similar
practice in their own professional context. In many cases, QAA was able to use the expertise and
networks of international reviewers to support its wider enhancement activities in Scotland.
The pilot has proved to be a very positive experience, bringing added depth and dynamism to
the review teams, with some logistical considerations (such as travel and visa arrangements) to
be further explored. In December 2012, the QAA Board reviewed the evaluation of the pilot in
Scotland and endorsed the inclusion of international reviewers in other QAA review methods in
future, where they can add real value in a meaningful way. This would begin with the new Higher
Education Review method.
QAA's review methodologies follow the four-stage model:
•
self-evaluation (including a student written submission)
•
peer review (including students)
•
site visit (including preparatory/briefing stages as well as the main review visit)
•
published report (including the provider's action plan).
Since QAA's last ENQA review in 2008, several methodological improvements have been made.
For example, all institutions, regardless of the outcome of their review, are now required to develop
and publish an action plan, or to submit a follow-up report that addresses all recommendations
and features of good practice highlighted in the report. This action plan should be developed in
consultation with the student body and should contain a link to the QAA report.
31
Student involvement in reviews has been enhanced. In methods where it is appropriate and
feasible, students submit a student written submission to sit alongside the provider's self-evaluation
document. The role of facilitator is also now mirrored by a lead student representative. Both of these
initiatives are intended to ensure that the student voice is heard clearly throughout the review and
to enhance students' opportunities to be involved in external quality assurance processes.
In Scotland, rather than produce a separate student written submission, students are directly
involved in the development of the institution's self-evaluation document. Direct student input
at the self-evaluation stage will be included in the handbook for the new method of Higher
Education Review, which is currently under consultation (see 'Recent UK higher education policy
developments', page 9).
Because of the number of review methodologies carried out by QAA, the Agency is currently
working on a set of common principles for student participation in external quality assurance.
Enhancement continues to be central to QAA's review methodologies. In Scotland, 10 years of work
on Enhancement Themes is currently being celebrated.35 In England, Northern Ireland and Wales
there is now a formal judgement in this area, and a thematic element has been introduced to the
review method in England and Northern Ireland in order to facilitate system-wide analysis on certain
topics each year. In 2010, enhancement themes were introduced in Wales as part of the framework
to enhance the student experience. In Scotland, there is a searchable database linked to the work
on enhancement themes. QAA also now publishes searchable databases of features of good practice
and the recommendations highlighted in reports.
35
32
2013 Enhancement Themes Conference, Scotland: www.enhancementthemes.ac.uk/enhancement-themes-conference
Standard 2.5
Reporting
STANDARD
Reports should be published and should be written in a style which is clear and readily
accessible to its intended readership. Any decisions, commendations or recommendations
contained in reports should be easy for a reader to find.
Guidelines
In order to ensure maximum benefit from external quality assurance processes, it is important
that reports should meet the identified needs of the intended readership. Reports are sometimes
intended for different readership groups and this will require careful attention to structure,
content, style and tone.
In general, reports should be structured to cover description, analysis (including relevant
evidence), conclusions, commendations and recommendations. There should be sufficient
preliminary explanation to enable a lay reader to understand the purposes of the review, its form,
and the criteria used in making decisions. Key findings, conclusions and recommendations should
be easily locatable by readers.
Reports should be published in a readily accessible form and there should be opportunities for
readers and users of the reports (both within the relevant institution and outside it) to comment
on their usefulness.
How QAA meets this standard
Efforts have been made by QAA over the last two years to develop a clearer style of reporting.
The public report for each QAA review is now shorter and written in more direct language.
QAA staff have been trained to write clearly and to move away from the rather opaque reporting
style of the past. A review of QAA's communication unit has been undertaken since the last
ENQA review; the result was the creation of a Public Engagement Group, within which there is a
Multimedia Team. Members of the Multimedia Team work with QAA officers to ensure that reports
are clear and transparent.
Guides are published to help students, both UK and international, in their understanding of what
QAA does and how they can become involved. QAA has a dedicated Student Engagement Team
which is responsible for all aspects of student involvement in external quality assurance.
Guidance is also issued for higher education providers, along with online briefings for reviews.
Facilitators and lead student representatives for providers are invited to attend face-to-face
briefing sessions.
Reports for Welsh higher education providers, as well as other documentation and correspondence,
are translated into Welsh.
Since its last review, QAA has published a glossary36 of commonly used quality assurance and
enhancement terms. An analysis of improved readability in QAA documents was carried out in 2012
and there are plans to carry out some further analysis over the coming year, beginning with work by
the Student Engagement Team with the QAA Student Advisory Board.
36
QAA glossary: www.qaa.ac.uk/aboutus/glossary/pages/default.aspx
33
QAA believes that its work on the use of multimedia, particularly social media, to reach the wider
public is a feature of good practice.
Feature of good practice
Multimedia/social media
All QAA review reports (apart from those for the degree-awarding powers process) are published
on its website. Review reports all include judgements (the format of which depends on the
methodology used), features of good practice and recommendations for improvement.
These key findings are also highlighted in the way the report is presented on the website.
Today, QAA has an established and well accessed presence on social media channels: postings on
Twitter, films and animations on YouTube, and photos on Flickr. QAA continues to build its social
media portfolio to engage audiences, most recently on blogs, Facebook, Storify and ScoopIt.
In addition, QAA uses films and podcasts on its website, as additional channels to reach its
different audiences.
QAA podcasts: www.qaa.ac.uk/publications/podcasts
QAA films: www.qaa.ac.uk/publications/films
QAA continues to build relationships with, and link to and from, high-traffic websites and media
outlets used by higher education applicants, current students and other public audiences
(such as the UK Unistats and UCAS websites).
34
Standard 2.6
Follow-up procedures
STANDARD
Quality assurance processes which contain recommendations for action or which require
a subsequent action plan, should have a predetermined follow-up procedure which is
implemented consistently.
Guidelines
Quality assurance is not principally about individual external scrutiny events: It should be
about continuously trying to do a better job. External quality assurance does not end with the
publication of the report and should include a structured follow-up procedure to ensure that
recommendations are dealt with appropriately and any required action plans drawn up and
implemented. This may involve further meetings with institutional or programme representatives.
The objective is to ensure that areas identified for improvement are dealt with speedily and that
further enhancement is encouraged.
How QAA meets this standard
All QAA reports clearly state the outcomes of each review (judgements) and highlight features of
good practice and recommendations. Depending on the review method, the recommendations
may be described as 'essential', 'advisable' or 'desirable', or they may have a timeframe indicating
the urgency attached to them.
All higher education providers are required to produce an action plan37 or follow-up report setting
out their planned action against each of the recommendations, and also how they propose to build
on the features of good practice. The action plan is monitored by QAA; the level and frequency of
monitoring will depend on the outcome of the review.
A review is 'signed off' on publication of the action plan (if the outcome was positive) or at a point
during the monitoring of the action plan when it is clear that the matters raised by the review team
have been satisfactorily dealt with. At this stage, the higher education provider may use the
QAA Quality Mark on its website to indicate that it has met UK expectations.
37
E xamples of action plans and statements prepared by providers following review can be found at:
www.qaa.ac.uk/institutionreports/reports/pages/elir-year-on-university-of-Glasgow-11.aspx
www.chi.ac.uk/about-us/how-we-work/academic-quality
35
Standard 2.7
Periodic reviews
STANDARD
External quality assurance of institutions and/or programmes should be undertaken on
a cyclical basis. The length of the cycle and the review procedures to be used should be
clearly defined and published in advance.
Guidelines
Quality assurance is not a static but a dynamic process. It should be continuous and not ''once in
a lifetime''. It does not end with the first review or with the completion of the formal follow-up
procedure. It has to be periodically renewed. Subsequent external reviews should take into account
progress that has been made since the previous event. The process to be used in all external reviews
should be clearly defined by the external quality assurance agency and its demands on institutions
should not be greater than are necessary for the achievement of its objectives.
How QAA meets this standard
QAA review methods are normally based on a cycle of between four and six years. As a more
risk-based approach to quality assurance is embedded, a series of triggers is being developed;
so, increasingly, risk is being reflected in the length of cycle applied to each provider.
For example, in Wales, the review cycle is based on six years maximum, with the outcomes of a
review determining when the next one will be - this could be in two, four or six years. The ELIR
review method in Scotland now includes a process of annual monitoring, as does the educational
oversight method where the outcome of the monitoring process is one of the factors that determine
when the next review will take place.
Regardless of the method, the length of cycle and the type of follow-up reporting required, all
review methods scrutinise the internal quality assurance mechanisms used to assure the provider of
its own standards and quality.
Such mechanisms are usually based on periodic processes such as annual monitoring, external
examining and less frequent periodic reviews that provide a perspective over the longer term.
The combination of the internal and the external cycles allows providers to consider their
progress in response to recommendations and in the wider dissemination and/or embedding of
good practice.
In all cases, the relevant handbook for the review method clearly sets out the cycle and the
follow-up arrangements.
36
Standard 2.8
System-wide analyses
STANDARD
Quality assurance agencies should produce, from time to time, summary reports describing
and analysing the general findings of their reviews, evaluations, assessments, etc.
Guidelines
All external quality assurance agencies collect a wealth of information about individual
programmes and/or institutions and this provides material for structured analyses across
whole higher education systems. Such analyses can provide very useful information about
developments, trends, emerging good practice and areas of persistent difficulty or weakness
and can become useful tools for policy development and quality enhancement. Agencies should
consider including a research and development function within their activities, to help them
extract maximum benefit from their work.
How QAA meets this standard
The research and development function of QAA resides primarily within its Research, Development
and Partnerships Group, which has responsibility for analysing and disseminating the general
findings from reviews, in different formats appropriate to a range of audiences, including the
general public. The outputs from this activity include:
•
Outcomes38 and Learning from papers (based on review findings)
•
Talking about quality39 papers (opinion pieces)
•The Good Practice Knowledgebase40 (a searchable database on the QAA website, organised
around the learner journey and mapped to the Quality Code)
•The Recommendations Knowledgebase41 (a searchable database on the QAA website,
organised around the Chapters of the Quality Code)
•The Higher Education Empirical Research42 (HEER) database (summaries of higher education
reports and journal articles, for practitioners, managers, quality professionals, researchers and
policy makers in higher education).
QAA Scotland delivers a distinctive range of research and development activities for the higher
education sector in Scotland, for example through analysis of ELIR reviews and management of
Enhancement Themes. Key deliverables include a searchable database of theme-related resources,
annual overview reports commissioned by the Scottish Funding Council, and development work to
support the work of institutional quality managers across the Scottish higher education sector.
QAA has a Networks and Partnerships Team which runs the QAA Liaison Scheme. This scheme
provides a relationship between QAA and quality officers within QAA's subscribing higher education
providers, to promote enhancement activities and raise QAA's awareness of issues that affect the
higher education community. For example, the scheme provides the sector with an opportunity
to comment on the development and implementation of QAA policies and projects, such as the
Quality Code.
The scheme also helps to raise awareness among QAA staff of the challenges and issues facing
higher education providers. The Liaison Scheme does not operate in Scotland, where annual
discussion meetings are held with higher education providers as part of the Scottish review method.
QAA Outcomes papers: www.qaa.ac.uk/improvinghighereducation/pages/outcomes.aspx
38
QAA Talking about quality series: www.qaa.ac.uk/improvinghighereducation/debate/pages/default.aspx
39
QAA Good Practice Knowledgebase: www.qaa.ac.uk/improvinghighereducation/goodpractice/pages/default.aspx
40
QAA Recommendations Knowledgebase:
www.qaa.ac.uk/improvinghighereducation/recommendations/pages/default.aspx
41
HEER: http://heer.qaa.ac.uk
42
37
QAA review methods at a glance
Type of
provider
Country/ Method name and Commissioning
nation
abbreviation
body
England Institutional Review in
HEFCE
Northern England and Northern
DELNI
Higher education
Ireland (IRENI)
Ireland
institutions including
Enhancement-led
Scottish Funding
universities and other
Scotland Institutional Review
Council
degree-awarding
(ELIR)
bodies
Institutional Review
HEFCW
Wales
(Wales)
HEW
Professional,
statutory or
regulatory body
(PSRB)
UK-based
providers
UK
General Osteopathic
Council Review
(GOsC)
GOsC
England
Review of College
Higher Education
(RCHE)
HEFCE
Publicly funded
further education
Integrated Quality
colleges that provide Northern
and Enhancement
DELNI
programmes on
Ireland
Review (IQER NI)
behalf of UK degreeReview of Foundation
awarding bodies
HEFCW
Wales
Degrees in Wales
HEW
(FD Wales)
UK higher education
Providers of the
professional
UK
postgraduate
qualification for early
years educators
Independent higher
education providers,
including private
providers
Overseas
providers
UK
institutions
applying
to obtain
degreeawarding
powers or
university
title
UK
Providers that have a
formal agreement to
deliver courses for a UK
UK degree-awarding
body
Applicants for
Foundation Degree
awarding powers
(FDAP)
Applicants for taught
degree-awarding
powers (TDAP)
UK
Applicants for
research degreeawarding powers
(RDAP)
38
Any institution that
provides UK higher
education
UK
Handbook
QAA web pages
Handbook
QAA web pages
Handbook
QAA web pages
Handbook for
providers
Handbook for
visitors
QAA web pages
Handbook
QAA web pages
Handbook
QAA web pages
Handbook
QAA web pages
Handbook for
providers
Teaching Agency Handbook for
auditors
QAA web pages
Review for Educational Individual
Handbook
Oversight (REO)
QAA web pages
institutions,
in order to
Embedded College
comply with UK Handbook
Review for Educational
Border Agency
QAA web pages
Oversight (ECREO)
regulations
Early Years
Professional Status
Audit (EYPS)
Review of Overseas
Provision (Overseas)
QAA subscribers
QAA web pages
Scrutiny of
applications for
degree-awarding
powers or
university title
Department
for Business,
Innovation and
Skills (BIS) for
England, or
the relevant
government
ministry in
the devolved
administrations
QAA web pages
Concerns about
standards, quality or
public information
(Concerns)
QAA web pages
Any organisation
QAA Scotland
or individual
protocol
Applicants for
university title (UT)
Any
provider
Further
information
UK further education
Non-UK higher education
Type of
provider
UK-based
providers
Country/ Method name and Commissioning
nation
abbreviation
body
Providers delivering
higher education
programmes on
UK
behalf of awarding
bodies based outside
the UK
Recognition Scheme
for Educational
Oversight (RSEO)
Individual
institutions,
in order to
comply with UK
Border Agency
regulations
Handbook
QAA web pages
Department
for Business,
Innovation and
Skills (BIS)
Handbook
AVA licensing
criteria
Learning from
AVA review
2006-09
Access to HE
website
England
Validating agencies for the
Access to Higher Education
Diploma
Wales
Access Validating
Agency licensing,
monitoring and
relicensing
(AVA review)
Further
information
39
QAA’S COMPLIANCE WITH EUROPEAN
STANDARDS AND GUIDELINES (PART 3)
Standard 3.1 Use of external quality assurance procedures for
higher education (ENQA membership criterion 1)
STANDARD
The external quality assurance of agencies should take into account the presence and
effectiveness of the external quality assurance processes described in Part 2 of the
European Standards and Guidelines.
Guidelines
The standards for external quality assurance contained in Part 2 provide a valuable basis for the
external quality assessment process. The standards reflect best practices and experiences gained
through the development of external quality assurance in Europe since the early 1990s. It is
therefore important that these standards are integrated into the processes applied by external
quality assurance agencies towards the higher education institutions.
The standards for external quality assurance should together with the standards for external
quality assurance agencies constitute the basis for professional and credible external quality
assurance of higher education institutions.
How QAA meets this standard
QAA's quality assurance procedures, processes and guidelines, including the Quality Code,
take the Standards and Guidelines for Quality Assurance in the European Higher Education Area (ESG)
into account, as described in the previous section. The ESG is integrated into the processes QAA
applies in its external quality assurance of higher education institutions, ensuring that they are
robust, credible and professional.
See the table on page 26, which shows the relationship between the ESG and Quality Code.
40
Standard 3.2
Official status (ENQA membership criterion 2)
STANDARD
Agencies should be formally recognised by competent public authorities in the European
Higher Education Area as agencies with responsibilities for external quality assurance and
should have an established legal basis. They should comply with any requirements of the
legislative jurisdictions within which they operate.
How QAA meets this standard
QAA is the agency responsible for the external quality assurance of higher education in the
UK. Its role is recognised by all governments of the four UK nations, the funding councils and
representative bodies.
Since it was founded in 1997, QAA has provided continuous, rigorous external quality assurance for
higher education and responded to its growing remit. QAA has driven the evolution of audit and
review to ensure that methods remain fit for purpose in a rapidly changing external environment.
QAA complies with all relevant legal and regulatory requirements governing its operations as a UK
registered company and charity (see 'Corporate legal status', page 11).
41
Standard 3.3
Activities (ENQA membership criterion 1)
STANDARD
Agencies should undertake external quality assurance activities (at institutional or
programme level) on a regular basis.
Guidelines
These may involve evaluation, review, audit, assessment, accreditation or other similar activities
and should be part of the core functions of the agency.
How QAA meets this standard
QAA activities cover all aspects of the external quality assurance and quality enhancement of UK
higher education at institutional level.
All review activities are undertaken on a regular and continuous basis. Reviews are planned and
scheduled in accordance with the relevant methodology (see 'QAA review methods at a glance',
page 38), and to take into account new review methods or activities as QAA's remit expands
(for example, new providers entering the sector).
Scrutiny for degree-awarding powers is specific to each individual application and, therefore,
not repeated on a regular basis.
The Quality Code and external quality assurance activities form the core of QAA's functions.
Although QAA's review activities are at provider level, the Agency also produces supplementary
guidance43 and reference points, notably subject benchmark statements,44 to support providers at
programme level, or on specific matters such as skills for employability.45
QAA supplementary guidance: www.qaa.ac.uk/assuringstandardsandquality/pages/general.aspx
43
42
44
Subject benchmark statements: www.qaa.ac.uk/assuringstandardsandquality/subject-guidance/pages/subjectbenchmark-statements.aspx
45
Employability: www.qaa.ac.uk/assuringstandardsandquality/pages/employability.aspx
Standard 3.4
Resources (ENQA membership criterion 3)
STANDARD
Agencies should have adequate and proportional resources, both human and financial, to
enable them to organise and run their external quality assurance process(es) in an effective
and efficient manner, with appropriate provision for the development of their processes
and procedures.
How QAA meets this standard
QAA is confident that it has the financial and human resources and the infrastructure necessary to
manage its external quality assurance activities efficiently and effectively, and to facilitate continuous
improvement in its processes and procedures. This improvement includes the ongoing personal and
professional development of QAA's staff and reviewers, which are key resources.
QAA's external quality assurance activities are led by its Director of Reviews, Dr Stephen Jackson.
The Reviews Group has responsibility for the effective management of all review activity.
The Reviews Management Team consists of:
•
Deputy Director - responsible for the development of review activities, including the new
common review framework for England, and the organisation and delivery of review activities;
the Deputy Director is supported by various method coordinators
•
Head of Educational Oversight - responsible for delivery of reviews and annual monitoring
of independent providers requiring oversight for UK Border Agency purposes
•
Head of Concerns and QAA Wales - responsible for managing review activity and wider
policy in Wales, and investigating concerns raised about institutions and complaints
against QAA
•
Head of Degree-Awarding Powers and University Title - responsible for managing the
scrutiny process which advises government on applications from institutions
•
Head of Access to Higher Education - responsible for qualification development and
regulation of the Access to HE diploma
•
Operations Support Manager - responsible for scheduling and coordination of reviews,
financial performance and management of the Reviews Support Team.
The Research, Development and Partnerships Group is led by its Director, Dr Jayne Mitchell, and its
management team includes:
•
ead of Standards, Quality and Enhancement - responsible for development and maintenance
H
of the Quality Code, qualifications framework and subject benchmark statements
•
ead of Research, Intelligence and Enquiry - responsible for analysing evaluations of external
H
quality assurance activities and the production of analyses of outcomes across review activities
•
Head of Networks and Partnerships - responsible for international activities, including
the international strategy and policy developments relating to the quality assurance of
transnational education.
In addition, there is a pool of assistant directors and development officers who, although assigned
to particular groups for line management purposes, work across QAA to manage the individual
review processes and contribute to wider QAA policy development.
43
External quality assurance in Scotland is led by QAA Scotland Director, Dr Bill Harvey, working with:
•
Head of Reviews - responsible for leading the development and delivery of all review activities
within QAA Scotland, including recruitment and training of reviewers and annual reporting to
the Scottish Funding Council
•
Head of Enhancement - responsible for leading the development and delivery of all
enhancement activities within QAA Scotland.
Wider staff support specifically for external quality assurance activities comes from:
•
entral Resources Team - providing flexible and responsive administrative support across
C
QAA, including the administration of appeals
•
Multimedia Team - editing, designing, proofreading and publishing QAA reports
•
Public Relations Team - handling media enquiries and promote review findings
•
Student Engagement Team - supporting the recruitment and development of students for
review activities
•
Service Desk - providing remote IT support for reviewers and staff.
A QAA Performance Dashboard has recently been developed as part of the Agency's ongoing
commitment to the effective management of all aspects of its resources, operations and activities.
The dashboard (see screenshot below) sets out, in graphical form, current information about how
QAA is performing against key targets (see also compliance with Standard 3.8).
44
Agency-wide resource to support quality assurance activities
QAA has a dedicated Resources Group led by the Director of Resources, Douglas Blackstock.
This group has responsibility for the effective management of all resources, including:
•
finance and planning
•
corporate and legal affairs
•
business development
•
information management and infrastructure
•
human resources and organisational development.
Financial
QAA's total income for 2011-12 was £14.03 million, and reserves amounted to £4.49 million.
QAA is largely funded from subscriptions from higher education providers and through contracts46
with the UK higher education funding bodies.
In recent years, however, QAA has widened its subscriber base47 to include voluntary private and
further education college subscribers, and has contracted to undertake reviews for a large number
of non-publicly funded institutions.
QAA has a robust planning process, linked with its Strategy 2011-14 (see page 20), and publishes
its Planning Framework on the QAA website.48 The QAA Board has established a Finance and
Strategy Committee, and six-yearly financial forecasts and income generation targets have been set.
This will enable QAA to take a longer-term view when negotiating with the funding bodies, setting
subscription levels, and targeting opportunities for business development.
In 2012, QAA introduced a dedicated procurement and contract management function in order
to ensure that it achieves maximum value for money (VfM). This has already resulted in significant
financial savings as well as improved efficiencies in resource management. A VfM group, drawn
from across the Agency, champions wider VfM issues.
QAA has also pursued other sources of income. It has established a trading subsidiary,
QAA Enterprises,49 and an Innovation and Development Fund to develop new products and
services for the benefit of UK higher education.
Technical infrastructure
QAA has dedicated independent information technology systems to support external quality
assurance. It supports and maintains an Agency-wide operational database (QMIS) used to manage
reviews, projects and events, and has invested in upgrading its reviewer extranet (Qmmunity)
to enable reviewers and staff to easily access documents wherever they are working (including
overseas) and for QAA to more easily gather and analyse data.
Office premises
QAA has offices in four cities in the UK to support its activities: Gloucester is the main base, with a
dedicated office for QAA Scotland in Glasgow, and smaller bases in Cardiff (Wales) and London.
QAA contracts: www.qaa.ac.uk/aboutus/corporate/pages/contracts.aspx
46
47
QAA subscribers: www.qaa.ac.uk/aboutus/subscribing-institutions/pages/default.aspx
48
QAA Annual Plan 2012-13: www.qaa.ac.uk/aboutus/corporate/documents/annual-plan-2012-13.pdf
49
QAA Enterprises: www.qaa.ac.uk/aboutus/pages/qaa-enterprises.aspx
45
Human resources - staff and expert peer reviewers
QAA is committed to recruiting and retaining skilled and talented staff. QAA has around 170
staff who bring experience from within the higher education sector, from other professional
backgrounds, and from a range of national and international contexts.
During 2010-11, QAA undertook a review of its organisational structure and adopted a number
of changes to ensure that it has sufficient resources to continue to respond flexibly and swiftly to
changing and emerging requirements.
QAA has around 600 reviewers who are selected from a rich pool of talent and experience, both in
the UK and internationally. QAA's review processes are based on peer review - for a typical (IRENI)
visit, a team of four peer reviewers (including a student reviewer), supported by a QAA officer,
visits the higher education provider for up to five days.
QAA has made details of current reviewers available (with their permission) on its website.50
A searchable database gives key information about reviewers, including:
•name
•
highest qualification
•
current employer
•
current QAA role(s) across all methods
•
reviews undertaken
•involvement in other QAA activities where a final report will not be in the public domain
(for example, scrutiny of applications for degree-awarding powers or university title).
Staff and reviewer development and performance management
QAA has a strong commitment to staff development, and invests in their training and development
at all levels of the organisation.
All internal staff have a comprehensive induction programme when they join the Agency and are
supported by a mentor from outside their own group. Members of staff are able undertake relevant
academic, vocational or professional qualifications with the support of QAA, and are encouraged
to be involved in at least one cross-agency activity. In addition, QAA holds regular staff information
talks covering a wide range of topics (including presentations from other European quality assurance
agencies) and these are open to all employees.
Performance management is an integral part of QAA processes, both informally though feedback
and formally through annual performance reviews for staff. The QAA performance review process
provides an opportunity for staff and their managers to reflect on past performance, to agree targets
and objectives, and to identify training and development needs for the coming year. In 2012, QAA
introduced a new annual performance review mechanism which will be piloted in 2013.
QAA also asks reviewers, officers and higher education providers for feedback on the performance
of reviewers during review. QAA has proposed a more targeted system, where reviewers reflect on
their own performance during review and provide feedback on the other members of their review
team. QAA made this proposal as part of the 2012 QAA Reviewer Survey. Most respondents to that
survey agreed with the principle of a more targeted system. QAA is now revising the proposals with
a view to implementing the new system by the beginning of the next academic year (2013-14).
50
46
QAA reviewers: www.qaa.ac.uk/institutionreports/our-reviewers/pages/default.aspx
Standard 3.5
Mission statement (ENQA membership criterion 4)
STANDARD
Agencies should have clear and explicit goals and objectives for their work, contained in a
publicly available statement.
Guidelines
These statements should describe the goals and objectives of agencies' quality assurance
processes, the division of labour with relevant stakeholders in higher education, especially the
higher education institutions, and the cultural and historical context of their work. The statements
should make clear that the external quality assurance process is a major activity of the agency
and that there exists a systematic approach to achieving its goals and objectives. There should
also be documentation to demonstrate how the statements are translated into a clear policy and
management plan.
How QAA meets this standard
QAA has clear and explicit goals and objectives for its work, and these are contained in a publicly
available statement.51 QAA's overall purpose is clearly stated on its website:
We are the Quality Assurance Agency for Higher Education (QAA).
Our mission is to safeguard standards and improve the quality of
UK higher education.
QAA's corporate governance structure is publicly available on its website, accompanied by the QAA
Memorandum of Association, Articles of Association and other formal documents.52 The website
provides information about how the Board develops and oversees QAA's strategic direction, and
explains the responsibility of the Chief Executive and Directors for directing and managing the
organisation (see also Introduction, pages 4-10).
During QAA's annual planning process, the QAA Strategy 2011-14 (see page 20) is used to establish
annual priorities, and these are incorporated into group and team service delivery statements,
ensuring that there is a strong focus on QAA's four key aims (stated in the Strategy) throughout the
organisation. The service delivery statements, in turn, are the basis for setting individual objectives
during annual performance reviews, so that all colleagues can contribute to achieving the aims.
The QAA Framework for Performance Management, together with the Performance Dashboard
(see page 44), enables systematic monitoring of progress against the objectives (see also compliance
with Standard 3.8).
About QAA: www.qaa.ac.uk/aboutus/pages/default.aspx
51
52
QAA corporate governance: www.qaa.ac.uk/aboutus/corporate/pages/default.aspx
47
Standard 3.6
Independence (ENQA membership criterion 5)
STANDARD
Agencies should be independent to the extent both that they have autonomous
responsibility for their operations and that the conclusions and recommendations made in
their reports cannot be influenced by third parties such as higher education institutions,
ministries or other stakeholders.
Guidelines
An agency will need to demonstrate its independence through measures, such as:
•
its operational independence from higher education institutions and governments is guaranteed in official documentation (e.g. instruments of governance or legislative acts)
•
the definition and operation of its procedures and methods, the nomination and
appointment of external experts and the determination of the outcomes of its quality assurance processes are undertaken autonomously and independently from governments, higher education institutions, and organs of political influence
•
relevant stakeholders in higher education, particularly students/learners, are consulted in the course of quality assurance processes, the final outcomes of the quality assurance processes remain the responsibility of the agency.
How QAA meets this standard
QAA has strong links with sector bodies and other stakeholders in the higher education sector,
both in the UK and internationally, as described in the introduction to this report. Government and
funding councils have an influence on the scale and volume of QAA work including, for example,
new institutions to be reviewed. This does not, however, impinge on QAA's independence,
as defined in the ENQA standard.
QAA is an independent body with autonomous responsibility for its operations. Its external quality
assurance processes and judgements are driven by criteria and methodologies which cannot be
influenced by third parties. It is established as a company limited by guarantee, and is a registered
charity. QAA has no constitutional links with government, or with individual higher
education providers.
As a registered charity, QAA has responsibilities under charities legislation - the Charities Act 2011
- to provide benefit to the public and is required to produce an annual report outlining how it is
achieving this.53 The Charity Commission, the regulator for charities in England and Wales, makes it
clear that charities must be independent from governmental authorities:
... [a charity] must exist in order to carry out its charitable purposes, and not for the purpose of implementing the policies of a governmental authority, or of carrying out the directions of a governmental authority.54
QAA's key objectives are set out in official company documentation (the QAA Articles of Association55).
Charities Act 2011: www.legislation.gov.uk/ukpga/2011/25/introduction and www.legislation.gov.uk/asp/2005/10/
contents
53
The Charity Commission: www.charitycommission.gov.uk/publications/rr7.aspx
54
QAA Articles of Association: www.qaa.ac.uk/aboutus/corporate/pages/default.aspx
55
48
QAA's strategy is determined by its Board (see Introduction, pages 4-10), which is responsible
for ensuring that effective management is in place to deliver on that strategy. The Board is also
responsible for approving all QAA's key procedures and methods.
QAA consults widely whenever there are changes to a review method, or changes are made to
the Quality Code. However, the final decisions on such changes lie with QAA.
With respect to QAA teams who conduct reviews of higher education providers leading to
judgements, the judgements made lie solely with the team, and are entirely driven by the processes
and criteria specified in the relevant review methodology, free from the influence of third parties.
QAA has full responsibility for the appointment of the external experts appointed to the teams
and for the final outcomes of its quality assurance processes. QAA's selection criteria for reviewers
include mechanisms to avoid conflicts of interest (see compliance with Standard 3.8) as part of
the process of reinforcing the independence of the judgements reached.
49
Standard 3.7 External quality assurance criteria and processes used
by the agencies (ENQA membership criterion 6)
STANDARD
The processes, criteria and procedures used by agencies should be pre-defined and publicly
available. These processes will normally be expected to include:
•
a self-assessment or equivalent procedure by the subject of the quality assurance process
•an external assessment by a group of experts, including, as appropriate, (a) student member(s), and site visits as decided by the agency
•
publication of a report, including any decisions, recommendations or other formal outcomes
•a follow-up procedure to review actions taken by the subject of the quality assurance process in the light of any recommendations contained in the report.
Guidelines
Agencies may develop and use other processes and procedures for particular purposes.
Agencies should pay careful attention to their declared principles at all times, and ensure both
that their requirements and processes are managed professionally and that their conclusions and
decisions are reached in a consistent manner, even though the decisions are formed by groups of
different people.
Agencies that make formal quality assurance decisions or conclusions which have formal
consequences should have an appeals procedure. The nature and form of the appeals procedure
should be determined in the light of the constitution of each agency.
How QAA meets this standard
All review processes, criteria and procedures used by QAA are pre-defined and publicly available on
the QAA website.56 Publication of new procedures is accompanied by electronic communications,
briefings and training events so that they are clearly understood.
All of QAA's processes include the requirement for a self-assessment in the form of a self-evaluation
document, through which the higher education provider is expected to reflect on its management
of the quality and standards for which it is responsible. In addition, most QAA review methods now
encourage a submission by students.
All of QAA's reviews are based on a peer review process involving site visits and are undertaken by
a group of experts. Each review results in the publication of a report that states the review team's
judgements, makes recommendations and identifies features of good practice.57 Higher education
providers are required to develop and maintain an action plan, demonstrating how good practice
is maintained and disseminated, as well as addressing issues raised in recommendations.
This provides the basis for following up and monitoring progress. QAA also publishes outcomes
papers, overview reports and summaries of good practices and recommendations (see also
compliance with Standard 2.8).
QAA reviews: www.qaa.ac.uk/institutionreports/types-of-review/pages/default.aspx
56
57
50
QAA review reports: www.qaa.ac.uk/institutionreports/pages/default.aspx
The higher education context in the UK and internationally is changing rapidly, with a greater
diversity of providers of higher education and new methods of delivery, often involving partnerships
that cross international boundaries. QAA recognises the need to be able to respond swiftly and
effectively to such developments, while maintaining rigorous and robust quality assurance for
UK higher education, wherever and however it is delivered. From time to time, this results in the
need for new procedures, which are developed in discussion with stakeholders and their partners
in the UK and internationally. Whatever review processes are used, QAA ensures that they are
professionally managed, and that conclusions and judgements are reached consistently and
independently. Judgements and outcomes are subject to scrutiny by an internal moderation panel in
order to ensure consistency of judgements.
QAA is conscious that judgements reached in its reports may have consequences for the providers
involved. Although QAA has confidence in its review processes, there may be times when a
higher education provider wishes to challenge a decision. QAA, therefore, has robust appeals and
complaints processes in place.58
QAA has clear published procedures for responding to complaints from higher education providers
and for handling appeals against specific decisions. QAA distinguishes between complaints
and appeals.
•A complaint is an expression of dissatisfaction with services provided by QAA or actions it
has taken.
•
An appeal is a challenge to a specific decision, in specific circumstances.59
Complaints or concerns about higher education providers are referred to as 'Concerns'.
The processes for investigating complaints and appeals, tailored to each review method, are publicly
available and can be found the QAA website.
Since its last ENQA review in 2008, QAA has widened the range of judgements that can be
subject to appeal. Appeal outcomes are also now published on the website.60
Complaints to QAA: www.qaa.ac.uk/complaints/pages/default.aspx
58
59
Appeals to QAA: www.qaa.ac.uk/complaints/pages/appeals.aspx
60
See the following examples: www.qaa.ac.uk/institutionreports/reports/documents/midlands_int_appeal.pdf and
www.qaa.ac.uk/institutionreports/reports/documents/manchesterhe_mt-appeal.pdf.
51
Standard 3.8 Accountability procedures
(ENQA membership criterion 7)
STANDARD
Agencies should have in place procedures for their own accountability.
Guidelines
These procedures are expected to include the following:
1. A published policy for the assurance of the quality of the agency itself, made available on
its website.
2. Documentation which demonstrates that:
­
- the agency's processes and results reflect its mission and goals of quality assurance
­
- the agency has in place, and enforces, a no-conflict-of-interest mechanism in the work of
its external experts
­
- the agency has reliable mechanisms that ensure the quality of any activities and material
produced by subcontractors, if some or all of the elements in its quality assurance
procedure are subcontracted to other parties
­
- the agency has in place internal quality assurance procedures which include an internal
feedback mechanism (i.e. means to collect feedback from its own staff and council/
board); an internal reflection mechanism (i.e. means to react to internal and external
recommendations for improvement); and an external feedback mechanism (i.e. means to
collect feedback from experts and reviewed institutions for future development) in order to
inform and underpin its own development and improvement
­
- a mandatory cyclical external review of the agency's activities at least once every five years.
How QAA meets this standard
QAA is accountable to a wide range of internal and external stakeholders. It fulfils this accountability
through a comprehensive range of internal quality assurance mechanisms. In developing these
mechanisms, QAA has been an active participant in - and influenced by - the ENQA Internal Quality
Assurance Working Group (IQA).
Performance management and accountability
These mechanisms have been strengthened through the development and implementation
(in 2011-12) of the QAA Framework for Performance Management, which is published on the
QAA website.61 The framework provides an overarching quality assurance mechanism and helps to
embed a strong culture of performance management for the Agency. It consists of four key pillars:
Objective setting
and targets
Evaluation, monitoring
and performance
indicators
Performance review
and reporting
QAA Framework for Performance Management:
www.qaa.ac.uk/AboutUs/corporate/Policies/Documents/Framework-for-performance-management.pdf
61
52
Planning, budgeting
and resource
deployment
QAA accountability has also been reinforced through the introduction of group service delivery
statements and risk registers, which are monitored on a quarterly basis. QAA submits an Annual
Effectiveness Report62 to its Board.
QAA has an annual programme of internal audit in which an external organisation is commissioned
to scrutinise key areas of the Agency's work. These annual audits result in a report which includes
an action plan to address any areas of concern. QAA's Audit Committee agrees the annual internal
audit programme, receives audit reports, and checks that action plans are implemented.
Information security and accessibility
QAA has been certified for compliance with ISO/IEC 27001, the international standard setting
out the requirements for an information security management system.63 QAA conducts regular
information security audits.
The QAA website aims to meet the recommended standard of the World Wide Web Consortium for
XHTML 1.0, CSS and Level AA of the accessibility guidelines.64 The website also aims to be accessible
to assistive technologies and flexible for all users.
Policies
QAA policies65 (corporate, review, internal and human resources) and some strategies (for instance,
student engagement66) are publicly available. They help to assure the quality and transparency of
QAA's work by providing clear reference points. QAA's Information Publication Scheme67 reinforces
its commitment to openness and transparency. The policies are supported by QAA's values68 and
ways of working, which are the behaviours the Agency seeks to demonstrate in all that it does.
Avoiding conflicts of interest
QAA has mechanisms in place to ensure that those undertaking work on its behalf - both internal
and external - are fair and impartial in their work, and that conflicts of interest are avoided.
These mechanisms include:
•
ethical conduct and anti-bribery policy69
•
Single Equality Scheme70
•
whistleblowing policy71
•
Code of best practice for members of the QAA Board72
•
Registers of Board Members' and Directors' interests.73
QAA effectiveness: www.qaa.ac.uk/aboutus/corporate/board/documents/assessing_effectiveness_bd201159.pdf
62
63
ISO 27001: www.bsigroup.com/en-gb/iso-27001-information-security
64
Accessibility: www.w3.org/tr
65
QAA policies: www.qaa.ac.uk/aboutus/corporate/policies/pages/default.aspx
66
QAA Student Engagement Strategy: www.qaa.ac.uk/partners/students/student-engagement-qaa/pages/studentengagement-strategy.aspx
67
QAA Information Publication Scheme: www.qaa.ac.uk/publications/informationpolicyscheme/pages/default.aspx
68
QAA values: www.qaa.ac.uk/aboutus/strategy11-14/pages/values.aspx
69
QAA ethical conduct: www.qaa.ac.uk/aboutus/corporate/policies/documents/ethical_conduct.pdf
70
QAA Single Equality Scheme: www.qaa.ac.uk/publications/informationandguidance/pages/single-equality-scheme.aspx
71
QAA whistleblowing policy: www.qaa.ac.uk/aboutus/corporate/policies/documents/whistleblowingpolicy.pdf
72
QAA Code of best practice for members of the QAA Board: www.qaa.ac.uk/publications/informationandguidance/pages/
code-best-practice-qaa-board.aspx
73
QAA register of interests: www.qaa.ac.uk/aboutus/corporate/board/pages/register-interests.aspx and
www.qaa.ac.uk/aboutus/corporate/pages/directorate.aspx
53
In 2010, the UK Bribery Act74 provided an opportunity for QAA to review and update its policies in
relation to conflicts of interest. The revised Ethical Conduct and Anti-Bribery Policy applies to all
employees, Board and Board Committee Members, temporary workers, consultants, contractors,
agents and subsidiaries acting on behalf of QAA in the UK and overseas.
QAA's human resources policies have been updated to provide guidance for staff wishing to
undertake work outside the Agency (paid or unpaid), to ensure that there is no conflict of interest or
risk to QAA's reputation.
All QAA reviewer contracts include a Code of Practice, and copies of the Ethical Conduct and
Anti- Bribery Policy, to prevent conflicts of interest. The QAA reviewer selection processes are
designed to identify and screen out potential conflicts of interest, and to achieve a balance in review
teams. QAA reviewer training covers equality, diversity and the avoidance of conflicts of interest,
and reviewers are asked to declare any interests before being allocated to a review. Details of the
review team are sent to the higher education provider in advance, enabling the provider to draw
attention to any anticipated conflicts of interest or other concerns.
Equality and diversity
As part of QAA's commitment to equality and in light of recent UK legislation (the Equality Act
2010),75 QAA developed its own Single Equality Scheme in close consultation with staff and with
external stakeholders. The Scheme sets out clear, achievable goals and aspirations to promote
equality and diversity, in terms of both internal procedures and engagement with the wider higher
education sector. QAA has recently published its first annual report on the scheme, together with an
updated action plan.76
QAA has three times successfully achieved Investors in People recognition.77 Launched in 1991,
Investors in People is the UK's leading people management standard. It specialises in transforming
business performance through people. In 2010, QAA achieved Bronze status in the Investors in
People scheme, and will be reviewed again in 2013.
Subcontractors
Where subcontractors are involved in QAA activity, whether as reviewers or in any other capacity,
their terms of reference or engagement are set out in a formal contractual agreement against
which performance can be managed. QAA peer review processes are designed to incorporate
quality assurance of the process and its outputs (including those of external reviewers) without
compromising the process itself.
All QAA publications are subject to rigorous and robust editing and review including, where
appropriate, the use of external contracted editors. The writing and editing process includes input
from external editors as well as from QAA reviews staff and the in-house Multimedia Team.
Quality assurance is also supported by relevant internal procedures. These include:
•
levels of authority and approval processes for contracts
•
the appointment of internal method coordinators
•
regular moderation meetings
•
formal systems for updating electronic information
•
sign-off and approval processes for publications.
74
UK Bribery Act: www.legislation.gov.uk/ukpga/2010/23/contents
Equality Act 2010: www.legislation.gov.uk/ukpga/2010/15/pdfs/ukpga_20100015_en.pdf
75
QAA (2012) Single Equality Scheme: First Annual Report: www.qaa.ac.uk/publications/informationandguidance/pages/
single-equality-scheme-first-annual-report.aspx
76
54 77 Investors in People: www.investorsinpeople.co.uk
Feedback and reflection mechanisms
QAA actively encourages internal and external feedback, in order to inform development and
improvement of its work.
A confidential annual staff survey, managed by an external body, provides a route for staff to raise
any concerns and is a key internal feedback mechanism. The results of the 2012 annual survey have
resulted in action plans to address issues raised, which are reviewed regularly and build on good
practice identified.
Internal reflection mechanisms also include an annual staff conference; scheduled Board,
Directorate, group and team 'away days'; and short, informal meetings and cross-Agency groups.
QAA also holds monthly staff briefings, where members of staff have the opportunity to raise
questions (anonymously if they prefer).
There are a number of opportunities to reflect on, and respond to, internal and external
recommendations for improvement, and to consider how to respond to, or influence, the changing
context in which QAA works.
QAA values external feedback on all aspects of its work, both retrospectively and when planning for
future developments. QAA has a number of established external feedback mechanisms.
•QAA has a Student Advisory Board, a Research Strategy Advisory Board and other groups of
external experts, to whom it turn for views.
•All participants in QAA events, including training, consultation events, briefings or conferences,
are encouraged to provide feedback. QAA routinely analyses such feedback to capture
lessons learned.
•Consultation events, such as those for the development of the Quality Code, exemplify the
range of mechanisms and opportunities - electronic and face-to-face - that are used to facilitate
both professional input and feedback about the consultation process itself.
•Other examples include the use of focus groups, blogs, discussion boards and social media,
including LinkedIn, Twitter and Facebook.
QAA welcomes external scrutiny and review, and the opportunity to develop in line with
recommendations made, and seeks to adopt relevant external standards.78 See pages 22-24 for
QAA's actions and progress, following its 2008 review by ENQA.
QAA external standards: www.qaa.ac.uk/aboutus/corporate/pages/standards.aspx
78
55
COMPLIANCE WITH ENQA MEMBERSHIP
CRITERION 8 (MISCELLANEOUS)
The Agency pays careful attention to its declared principles at all times, and ensures both that its
requirements and processes are managed professionally and that its judgements and decisions are
reached in a consistent manner, even if the judgements are formed by different groups.
See compliance with Standards 2.3 and 2.4.
If the Agency makes formal quality assurance decisions, or conclusions which have formal
consequences, it should have an appeals procedure. The nature and form of the appeals procedure
should be determined in the light of the constitution of the agency.
See compliance with Standard 3.7.
The Agency is willing to contribute actively to the aims of ENQA.
QAA was a founding member of ENQA, and has contributed to ENQA's aims and activities in
various ways:
•
through membership of the ENQA Board
•
through membership of ENQA project groups
•
every QAA Director is a member of an ENQA working group
•
QAA Assistant Director Fiona Crozier held the vice-presidency of ENQA
•QAA has also been involved, at Director level, in the standing Internal Quality Assurance (IQA)
working group, which has directly affected the development of QAA's own Performance
Management Framework
•through its ENQA involvement, QAA aims to share and disseminate knowledge, and to learn
from the working practices of other agencies.
56
DEVELOPMENT OF THE QAA
SELF-EVALUATION REPORT
As part of the development of the 2013 self-evaluation report, QAA has undertaken extensive staff
and wider stakeholder engagement to critically reflect on its progress since its review by ENQA
in 2008, its compliance with the Standards and Guidelines for Quality Assurance in the European
Higher Education Area (ESG), current challenges and areas for future development. All of the input,
expertise and advice gathered from these stakeholders have been used to inform and directly
influence this self-evaluation report.
QAA established an internal project team for its 2013 review by ENQA, responsible for overseeing all
activity related to the review. This included primary responsibility for the development and drafting
of the report, and managing the stakeholder engagement process. The project team reported to a
project board.
Below is a table detailing the stakeholder engagement programme undertaken as part of the
development of this self-evaluation report.
Stakeholder
group
QAA Board
Date
1 October
2012
Type of
activity
Workshop
Number of
stakeholders
Key points raised
attending
22
Strengths of QAA
• Involvement of student reviewers
• Public information improved
• Quality Code underpins all methods
• Enhancement strengthened
•Move to common review framework
for the UK
Areas for development
•Adapting to the needs of the
devolved nature of UK higher
education
•Securing greater financial
independence through balanced
funding and income generation
•Expanding the use of international
reviewers on QAA review teams
(taking into account the need to
add value)
57
Stakeholder
group
58
Date
QAA staff
31 October
2012
QAA staff
31 October
2012
Professional,
statutory
and
regulatory
bodies
(PSRB)
survey
November
2012
Number of
stakeholders
Key points raised
attending
14
Workshop
•Maturity of QAA and the UK higher
on ESG 2
education sector
compliance
•QAA review methods have changed
several times since 2008
•UK context is complex - especially
the number of review methods
•Political influences have a bearing on
the nature of quality assurance and
enhancement activities
•There are common principles to QAA
reviews but diversity in QAA’s work
•QAA guidance for institutions and
reviewers is clear
•Rapid development is often
required - for instance, in getting
the educational oversight method
operational for 2012
•Availability of reviewers is a
constraint in team composition
•More feedback for reviewers on their
report writing
16
Workshop
•Strength of QAA's reviewer
on ESG 3
experience, credibility and training
compliance
•Potential to develop quality assurance
and
qualifications for reviewers?
discussion
• Strong internal feedback mechanism
about the
•Relatively small percentage of
art of
students seen during reviews and
self-reflection
not all students see the benefits of
outcomes of quality assurance in
their student lifetime - opportunities
for more innovative involvement
in future?
•Opportunity to look forward and
plan more than one year at a time
Type of
activity
Online
survey
Responses
from six
PSRBs
•Increased communication would be
appreciated - alerts on consultation/
increased data sharing
•QAA publications generally found to
be useful
•66% of respondents actively use
QAA reports and findings
•Setting of national benchmarks and
leading enhancement are valued
•Perception that review criteria are
not necessarily applied equally across
the UK
Stakeholder
group
Date
Type of
activity
QAA Student 16
November
Advisory
2012
Board
Workshop
19
November
2012
Workshop
QAA
Sounding
Board
Number of
stakeholders
Key points raised
attending
17
•NUS research shows students are
confident in quality assurance of
higher education providers, though
currently most don't know how
this works
•Brand recognition - QAA should be
as well known as Ofsted for those
considering higher education
•Are the differences in review
methods across nations positive
or negative? Are lessons being
transferred across?
•Does the student reviewer have the
capacity to examine all judgements?
5
•Independence is a key factor for
the sector
•UK context has moved ground in the
last five years, with QAA developing
greater sensitivity in response more devolution potentially means
more independence; devolution is
fact, not risk
•International reviewers - response
needs to be proportionate; context
should be internationalising QAA,
not reviews
•Progress still to be made on not
seeing England as the standard,
and other countries as variants
• Public confidence has improved
•Importance of retaining
organisational focus
59
Stakeholder
group
QAA
Scotland
60
Date
20
November
2012
Type of
activity
Workshop
Number of
stakeholders
Key points raised
attending
9
Focused on the issues which
were highlighted as concerns and
recommendations in 2008 ENQA
review of QAA, including:
•international reviewers - this has
been in place in Scotland for five
years, with advantages in being able
to ensure that for each review, the
team and the review process has an
outward-looking focus
•public information - Scotland has
had a positive experience with the
development of short, summary
reports of review outcomes, and
QAA generally has made great
strides in this area
•review cycles - QAA operates different
review cycles in different parts of the
UK, but under a common framework
- the Quality Code - and with
comparable judgements
•appeals - the process has been
changed to allow appeals of limited
confidence judgements across the UK
•student reviewers - QAA has a
longer period of experience of this
in Scotland but QAA as a whole has
fully addressed this issue, not just
in review team membership but in
relation to student engagement in
general; indeed QAA systems now
represent a model of good practice.
QAA: CURRENT CHALLENGES AND AREAS FOR
FUTURE DEVELOPMENT
A diverse UK context
One of the strengths of higher education in the UK is its diversity. In particular, that the devolved
governments of Scotland, Wales and Northern Ireland have responsibility for their own education
policies. QAA manages this diversity of needs through different types of review which meet the
requirements of the four nations of the UK. Our challenge is to do this flexibly, ensuring that the
interests of each country are met, within the context of an overall UK higher education framework.
We do this by ensuring all of our methods are underpinned by the Quality Code. The transition to the
Quality Code will be completed in 2013-14, and we will formally launch a Quality Code Portal on our
website, to aid institutions and the public in understanding the Quality Code's role and relevance.
A common review framework for England
QAA is moving to a common framework for more risk-based review in England from 2013-14.
Many of our reviewers already work across different review methods, and it is intended that the
framework will bring greater consistency in review, regardless of the type of provider reviewed.
This transition needs careful handling and, to facilitate it, there will be an annual 'super-conference'
to which all reviewers will be invited, and where they can share their experiences.
Diversity in institutional subscribers
In recent years, QAA has seen a steady growth in voluntary subscribers - higher education providers
from both the private and further education college sectors.
QAA has also welcomed the 2012 Lingfield Report, Professionalism in Further Education,79 which
recommended a greater role for QAA, and we wish to deepen our relationship with the further
education sector.
QAA will also encourage as many as possible of the independent colleges that we review to convert
to a long-term subscriber relationship, confirming our role as the quality assurer of all higher
education provision in the UK, regardless of who delivers it.
These changes pose challenges for the governance of QAA, and also for how we allocate resources
and develop structures to manage relationships with both existing and new subscribers. We have
made a first step by including on our Board a member appointed by the further education sector.
We have also begun a programme to develop our subscriber services, tailored to the needs of
different providers.
The age of austerity
Although an independent charity, QAA is not immune to the pressures on public and institutional
finance. We have developed a finance strategy which sets out our aim of securing financial
independence through sustainable, diversified income streams, and includes a balanced approach,
encompassing both cost efficiencies and revenue generation (through our trading subsidiary
QAA Enterprises).
In this economic climate, QAA will need to continue to use resources flexibly and creatively.
A robust performance management and service culture will ensure we produce more from the
resources available to us.
BIS (2012) Professionalism in Further Education: www.bis.gov.uk/assets/biscore/further-education-skills/docs/p/12-1198professionalism-in-further-education-final
79
61
International activity
The internationalisation of review methods in an organisation conducting over 250 reviews each
academic year is not without challenges. Nevertheless, QAA has developed proposals to enable us
to do this.
QAA is developing a new approach to quality assuring transnational education, in order to provide
public assurance that wherever UK higher education is delivered, it will meet UK expectations in full
and be reasonably priced for students (in relation to UK-based programmes).
QAA will continue to play its part in European and international networks, and is increasingly being
invited to collaborate with, and advise, parties in other countries on both a developmental and a
commercial basis. QAA is mindful of the need to ensure we receive sufficient return to make this
work viable.
62
ADDITIONAL REFERENCE MATERIALS
The following reference materials have been provided separately for the ENQA review team in
printed and/or electronic format.
1
Letter from Secretary of State for Education and Employability, 13 March 1997
2
The UK Quality Code for Higher Education:
i)
A Brief Guide
ii)
All Parts and Chapters of the UK Quality Code for Higher Education
3
Examples of QAA review handbooks, covering all four nations of the UK:
i)
Institutional Review (England and Northern Ireland)
ii)
Enhancement-led Institutional Review (Scotland)
iii)
Institutional Review (Wales)
4
Code of best practice for members of the QAA Board
5
QAA Consolidated Appeals Procedure (2013)
6
QAA Companies House Certificate of Incorporation
63
GLOSSARY
QAA provides a full glossary on its website, explaining terms that are frequently used in its work
and publications:
www.qaa.ac.uk/aboutus/glossary
ACKNOWLEDGEMENTS
QAA would like to thank all those who have contributed their time and expertise to the
development of this report. They include:
The QAA project team for the ENQA review, responsible for development and drafting of the report,
and managing the stakeholder engagement process and ENQA review team visit:
Douglas Blackstock (Chair), Matthew Cott, Fiona Crozier, Rebecca Ditchburn, Jane Grey,
Lindsay Houghton, Dr Jon Levett.
The QAA Public Engagement Team for their work in the design, editing and production of the
final report, particularly Steve Walker, Jane Fairbairn, Rachel Beckett and Dave Thompson.
The QAA project board for their guidance: Anthony McClaran (Chair), Carolyn Campbell,
Dr Julian Ellis, Dr Bill Harvey, Dr Stephen Jackson, Richard Jarman, Dr Jayne Mitchell.
Other QAA readers who commented on the draft, including Dr Irene Ainsworth, Dr Adam Biscoe,
Dr Anca Greere, Dr Liz Halford, Will Naylor and Liz Rosser.
External readers and advisers: Jennifer Allen (HEFCE), Helen Bowles (GuildHE), Dr Kirsty Conlon
(Universities Scotland), Karen Jones (University of Wales, Newport), Dr Cliona O'Neill (HEFCW),
Lesley Sutherland (Scottish Funding Council).
The six professional, regulatory and statutory bodies who took part in a stakeholder survey to inform
the report.
Finally, the QAA Board, QAA Student Advisory Board, QAA Scotland Committee and QAA Wales
Committee, and all QAA colleagues who have provided much valued input, advice and guidance
throughout the process.
64
The Quality Assurance Agency for Higher Education
Southgate House
Southgate Street
Gloucester
GL1 1UB
Tel 01452 557000
Fax 01452 557070
Email enquiries@qaa.ac.uk
Web www.qaa.ac.uk