PERFORMANCE MEASUREMENT
IN HIGHER EDUCATION: A
MALAYSIAN SCENARIO
Bernardine Renaldo Wong
International University of Malaya-Wales
Paper presented at QS-MAPLE 2013
Johannesburg, South Africa
IUMW is the result of a unique
collaboration between two
renowned universities – the
University of Malaya (UM) and
the transformed University of
Wales / Trinity Saint David.
UM
- established 1905
UW/TSD - Royal Charter 1828
IUMW
- March 13, 2013
IUMW is situated on a lush
campus in the heart of Kuala
Lumpur.
IUMW Campus
Facilities include:
a library with access to online
resources,
student recreational areas, a
sports centre,
e-teaching and learning
facilities,
well equipped lecture rooms,
a 400-seater auditorium
DUAL AWARDS
IUMW students the option to apply for
Dual Awards, where one certificate
awarded by International University of
Malaya-Wales and the other from the
transformed University of Wales /
Trinity Saint David.
IUMW students may also have
opportunities to study at the
transformed University of Wales /
Trinity Saint David.
Malaysian Higher Education: Facts
Item
1967
2007
Public Universities
1
20
Private Universities & University-Colleges
0
33
International Branch Campuses
0
4
Private Colleges & HE Institutions
2
488
Professors (public universities only)
27
1400
4560 (398)
873,238 (45,888)
Foreign Students
n/a
47,928
Population aged 18-24
n/a
3,474,200
Students (postgraduates)
Accessibility: Population aged 19-24 enrolled at Tertiary Level
Year
Population
Enrolment
%
1970
1,420,687
8,633
0.6
1980
1,624,274
26,410
1.6
1990
2,028,100
58,286
2.9
2000
2,626,900
211,484
8.1
2005*
3,353,600
649,653
19.4
2007*
3,474,200
847,485
24.4
2011*
3,628,300
1,485,600
40.0
Note: * Aged 18-24
Source: Department of Statistics, MoE & MoHE Statistics
GOAL: 50% of population enrolled in tertiary education by 2020
Malaysian Higher Education: Enabling Legislations
Item
Year
Private Higher Educational Institutions Act
1996
Education Act
1996 & 2002
National Accreditation Board Act
1997
National Higher Education Fund Act
1997
Universities & University Colleges Act
1971 & 2012
Malaysian Qualifications Agency Act
2007
Colleges/Universities in
Malaysia (2012)
Public Universities
20
Community colleges
37
International Branch Campus
7
Polytechnics
21
University/University Colleges
36
Private Colleges
485
Total - 607
Enrolment in HEI: (International) & Local Students
Category\Year
2008
2009
2010
(50,679)
399,897
12.7%
(58,294)
426,083
13.7%
(62,705)
478,924
13.1%
(18,485)
400,849
4.6%
(22,456)
414,964
5.4%
(24,214)
438,566
5.5%
International Student %
(69,164)
800,386
8.6%
(80,750)
841,050
9.6%
(86,919)
917,490
9.5%
Private HE Enrolment %
50.0%
49.3%
52.2%
Private HE Institutions
International Student %
Public HE Institutions
International Student %
Total
Projected International Student Enrolment
Year
Number of International
Students
2012
( end January)
2012
93,000
2015
150,000
2020
200,000
100,000
Advantages of Malaysia as an International Student Destination
• 11th most preferred destination for international students to
pursue tertiary education by the Institute of International
Education (UNESCO, 2009). Over a third of international
students are from Middle East and Africa.
•14th best in terms of quality in education by the World
Economic Forum's Global Competitiveness Report
•Safest and most peaceful country in South East Asia, and fourth
in the Asia Pacific region behind New Zealand, Japan and
Australia (5th Edition of the Global Peace Index report, 2011)
TRANSFORMATION OF THE HIGHER EDUCATION SECTOR:
(NATIONAL HIGHER EDUCATION STRATEGIC PLAN, 2007)
Source: Ministry of Higher Education
English Will
Become the
Main Medium
Universities
Compete to
Attract the Best
Talents
High Mobility
of Students
and Staff
Centre of Education
Shifting to the East
GLOBAL
TRENDS IN
HIGHER
EDUCATION
High
International
Connectivity
Entrepreneurial
and Innovative
Budget Size Will
Determine Quality
and Success
Heavy
Investments
in R&D
Rapid Rise of Global
Universities in the East
& Emergence of Many
More Universities
Benefits of Performance Measurement
• Mechanism for accountability to stakeholders
• Aid to strategic planning for the university
Performance Measurement
• Neely (1994) and Gregory and Platts (2005)
have defined performance measurement as
the process of quantifying and performance
measures as a metric used to quantify the
efficiency and effectiveness of action
respectively.
• According to the famous
quote by Lord Kelvin (1889)
a mathematical physicist
and engineer “If you cannot
measure it, you cannot
improve it.”
Examples of
Performance Measures of University
Examples from the Malaysian Context
Examples from the Malaysian Context
Stakeholders’ Expectations
Stakeholders
Expectation
Employer
Profitability and Immediate Performance
University
Reputation, Ranking and Academic Research
Graduate
High Salary and Minimum Hours
Parents
Reputation and Peer Pressure
Government
Graduate Employability, National Agenda: Economic
Upliftment and Social Stability
Governmental Support to Public Universities’ OPEX
Country
Malaysia
Singapore
Australia
Jordan
Indonesia
Thailand
Philippines
Taiwan
United Kingdom
Percentage
80%
70%
20%
30%
30%
30%
40%
40%
20%
University of Malaya: Measuring
Achievement Under Autonomy (New KPIs)
•
•
•
•
•
•
•
•
Increase in Graduate Employability
Increase in High Impact Research
Increase in Quality Academic staff
Increase in number of Experienced Professor
Efficiency of Financial Management
Increase in internally Generated Income
Effectiveness in Institutional Governance
Strengthening Networking and
Internationalization
• Increase in Number of Post-Graduates
UM KPIs
Key Result Area
International/ National
Recognition – Enhance
university standing as an
internationally renowned
higher educational
institution
KPI
Description
QS Ranking
Quality and reputation relative to other universities
around the world - in accordance to the methodology
used by QS to drive the rankings
QS Asia University Ranking
Quality and reputation relative to other universities
around the world - in accordance to the methodology
used by AUR to drive the rankings.
RU Ranking
Quality and reputation relative to other Public
Universities.
SETARA Rating
Monitoring teaching and learning process.
Percentage of international postgraduate students to total number
of postgraduate students
Percentage of international students in the postgraduate
population of the faculty.
Percentage of foreign academic staff to total number of academic
staff
Foreign academic staff is academic staff that hold
foreign passport, served UM for a period of not less
than three (3) months
Number of Active MoUs
No. of active MoUs to date
Number of international student exchange programmes (inbound
programmes & outbound programmes)
Number of international student exchange programmes.
'inbound (i.b) & outbound(o.b)' activities of students
taking part in academic (lab attachment/ research/
industrial training etc) activities for a minimum period of
seven (7) days.
Key Result Area
Stakeholders
perception
- Recognized &
well regarded by
Stakeholders
High Quality
Graduate –
producing
outstanding all
rounded
graduates
KPI
Description
Number of programs evaluated by professional
bodies (Professional Program)
Total professional program received recognition
from national and international professional bodies
*Postgraduate =Total professional program received
recognition from international professional bodies
Number of programs evaluated by non-professionals
bodies (MQF Compliance)
Total program received recognition from nonprofessional bodies based on MQF Compliance
UM's Incoming Student Choices
UM’s position in the list of top undergraduate
students choices
Percentage (%) completion of PhD (< 4 years)
Completion of PhD within or less than 4 years (<
4yrs)
Percentage (%) completion of Masters (< 2.5 years)
Completion of Masters within or less than 2.5 years
(< 2.5yrs) excluding clinical programmes
Undergraduate student employability (employed
within 6 months of graduation)
Students receiving prestigious awards (UG & PG)
Employed Graduates are referred to graduates who
have secured job, waiting for a job placement,
furthering their studies and under going training to
enhance their skills.
Their status are captured at the point of their
convocation, normally it is about 3-6 months after
their final examination.
Percentage (%) of undergraduate & postgraduate
students receiving prestigious awards
Key Result Area
Excellent
Research
Output
(Innovation /
service) – high
impact
publications,
commercial setups, innovative
products, policy
inputs
KPI
Description
Number of ISI articles
No. of articles in cited (ISI) journal. ISI Web of
Science. (refer to articles, reviews & proceedings
papers published in journal indexed in ISI-WOS
Number of books / copyright
(No. of patents applied & copyright registered)
No of IPR or original writings. Original writing refers
to scholarly work or books written that are externally
refereed or published by reputable publisher.
Number of patent granted
Number of products and services commercialised
Number of recognitions and awards
Total no of new patent granted for the particular
year. (Only one patent for one
product is counted )
Total Number of products and services
commercialised
Total no of awards conferred by national and
international bodies Examples of awards and
recognition include fellowships and scholarships or
appointment as committee in associations or
professional bodies. It does not include medals
won at exhibitions except for special awards.
Key Result Area
Quality academic
programmes – quality
programmes, to
provide outstanding
teaching – learning
experiences and
teaching environment
(add more KPIs)
Efficient Support
Services – library,
student counselling,
health, transportation
& financial.
KPI
Description
Ratio of undergraduate students to
academic staffs
Ratio of students registered for undergraduate (enrolment) to
the academic staff (permanent & contract). Academic staff: all
teaching staff including (Professor, Associate Professor, Senior
Lecturer, Lecturer, Senior Research Fellow, Research Fellow)
Ratio of postgraduate (research and
mixed mode) students to academic staff
Ratio of postgraduate students to academic staff (permanent &
contract) . Academic staff: all teaching staff including (Professor,
Associate Professor, Senior Lecturer, Lecturer)
Average evaluation scores for courses
(except for Faculty of Medicine)
The teaching evaluation scores of courses/subjects taught on
appraisal programme
Percentage of titles from the course
proforma reading list available for an
academic session
Percentage (%) of titles from the course proforma reading list
available for an academic session : this will reflect on the
quality of print materials available for teaching & learning
Increased access to electronic
resources through the InterAktif Portal
Increased access to electronic resources through the InterAktif
Portal (Library) : this will reflect on the actual usage of electronic
resources for teaching, learning & research activities
Students satisfaction - Health &
Transportation Services
Evaluation by students on the main services and facilities
Students satisfaction -Financial
assistance
Organize Personal development
workshops
Evaluation by students on the main services and facilities
(HEP)- Financial assistance
Provide training courses to students of
entrepreneurship culture
Total number of training courses provided to students of
entrepreneurship culture
Total number of Personal development workshops- open to all
students
Key Result Area
Niche/ Cutting
edge Research
Programmes –
cutting edge and
high impact
research
KPI
Increase in non-Research University
(RU) research grants
Description
Percentage (%) Amount of non RU research grants received
(e.g. MAKNA, MSC; etc.
Size of research expenditure, minimum
attained per year
Percentage (%) of research grants expended or utilised per year
Amount of research grants per staff
Amount of research grants per staff for S&T and non S&T
research
Amount of research grants for
postgraduate students
Critical mass of academic staff involved
as Principal Investigator
Average research experience of faculty
staff
Number of postdoctoral
Amount (RM) of research grants to support research by
postgraduate students
Increasing the critical mass of staff involved as PI actively in
research
Average years of research experience of Academic Staffs in a
faculty.
Number of postdoctoral
Accredited labs
Accredited labs
Number HICoE recognized at the
national level
Recognizing the Centre of Excellence (CoE) as HICoE
No. of HIR (Percentage Fund)
Total number of HIR based on year
Percentage (%) of Tier 1, 2, 3 and 4 ISI
Articles
Tiers 1 , 2, 3 & 4
Key Result Area
KPI
Description
Effective
International &
Corporate
Relations &
services
Number of international collaboration
identified
Number of international collaboration identified
Number of international events and
promotions to date
Number of international events and promotions during the year
Number of public talks / lecture series /
outreach program
Total Number of public talks / lecture series / outreach program
during the year
Human Capital Attracts and
develop high
quality staff
Enhance staff
satisfaction &
loyalty
Number of offer issued for the Bright
Sparks Program
Number of offer issued for the Bright Sparks Program
Total training hours (profesional
development) attended by staff
(Academic & Non academic)
Total Training Hours / Total Non - Academic Staff / Year
(profesional development)
Total Training Hours / Total Academic Staff / Year
Information Capital -
Network and server availability
Network availability and server availability for UM Campus
Integration of existing applications
Integration of existing applications
Up to date and integrated
technology and information
system
Database network for
planning, monitoring and
reporting
Academic Staff with PhD & Equivalent
Academic Icon
Academic Staff means permanent staff and contract staff which
holds a doctorate degree (PhD), including the highest (D. Eng,
DBA, DED, DSc) and equivalent to PhD (DPharm, Clinical
Master & Master of Expertise).
Moved to human capital (total academic icon based on year)
Key Result Area
Asset Capital –
Preserves the comfort
of campus society and
users, creates an
effective management
of University's
properties and
implements cost
effective
development projects.
Organizational
Supportive
environment Enhanced leadership
and management
skills
KPI
Number of requests or complaints
attended to and completed with –
[Request or complaints via Sistem
Aduan Penyelengaraan (SAP) ]
Total yearly spending from overall
OCAR JPPHB
Description
(a) Minor repairs (Within 5 working days) - Repairs and
maintenance of a priority nature such as changing bulbs, door
knobs and other minor repairs.
(b) Medium repairs (Within 21 working days) - Repairs and
maintenance of a routine nature such as standard carpentry
repairs, brickwork, plastering and other repairs to external
buildings.
Services rendered under maintenance are covered by OCAR
JPPHB (Other Charges Annually Recurring)
(Total OCAR – Total Charges) x 100
Total OCAR
The ratio of the administration post to
‘Talent Pool’
Initiatives to create a structure of the* Future Leaders who will
ensure continuity and excellence in university
*- Deputy Vice-Chancellor
- Dean
-Deputy Dean
- Head of Department
- Director
- Non Academic Grade 48 Up & Post-Position Equivalent
Percentage of academic and non
academic staff level management &
professional in higher education
institutions who have completed training
needs by IPT Leadership Competency
Framework
Leadership Competency Framework for Higher Education
Institutions of Inclusive Aspects of Leadership Excellence
Transformation of Higher Education Institutions.
Number of leadership courses
Total number of leadership courses provided during the year
Multi-Talented Graduates
Plato’s Academy: Let None Ignorant of
Geometry Enter Here
One Size Fits All?
• Although there are remarkable progresses made
overtime in performance measures, not all
performance measures problems have been
solved. There is no “one size fit all” model for
performance measures.
Performance Indicators in Higher Education
Are all things that are meaningful
measurable,
and
are all measurables meaningful?
Goodhart’s Law
"When a measure becomes a
target, it ceases to be a good
measure."
e.g. If researchers’ quality is measured by the number
of papers produced, unnecessary numbers of lowquality papers start being churned out.
Conclusions
• The global demand for Higher Education is increasing and
Malaysia is well placed to be a regional hub of higher
education.
•The reputation and sustainability of a University is
dependent of its provision of high quality programmes at
manageable costs.
•Performance measurements are essential for accountability
and strategic planning.
•Performance measures must ultimately be aligned to the
furtherance of the Vision and Mission of the University
Thank you