Measuring Engagement and Impact
15 and 17 February 2016
Professor Aidan Byrne, Chief Executive Officer
Australian Research Council
Outline
• Overview of the Australian Research Landscape
• ERA and ‘Engagement and Impact’
• Trial measures
Web: arc.gov.au I Email: Communications@arc.gov.au
2015–16 Federal Budget—$429 billion
2015–16 Federal Budget—R&D $9.7 billion
Commonwealth Investment in R&D 2015–16 (%)
NB tax measures reflect money
foregone, not direct funding
Web: arc.gov.au I Email: Communications@arc.gov.au
ARC NCGP % funding by institution 2008–2015
IRU
ATN
Go8
Unaligned unis
& others
National Competitive Grants Programme
Area of box represents $$ funded over the period 2011–2015.
N.B. - Centres of Excellence, SRIs, not awarded in every year.
- ITRP and DECRA only funding since 2012
ARC NCGP funding by Fields of Research 2002–2014
ERA 2015—Outcomes
ERA 2015 results are available in: State of Australian
University Research 2015-2016 , Volume 1, ERA
National Report.*
ERA 2015 outcomes show Australia’s universities
continue to produce high-quality research across a
wide range of disciplines.
Overall research quality is improving and reflects
ERA’s role in focussing universities on research
quality.
Of the UoEs assessed in both ERA 2015 and the
previous round, ERA 2012, 56 per cent of these
maintained the same rating, while 29 per cent
improved their rating by one.
The ERA results show an extraordinary breadth of
outstanding research performance in Australia.
*Available at: www.arc.gov.au.
ERA 2012 to ERA 2015 Growth
There was steady growth in Australia’s university research
activity between the ERA 2012 and ERA 2015 rounds, including
increases in:

the total number of assessed UoEs, up 6% to 2460

the number of research outputs submitted, up 5% to 432,747

the number of research staff submitted, up 11% to 67,579

total number of patents, up 20% to 936.
Distribution of ERA 2015 ratings across all
Australian universities
UoE ratings (aggregated four-digit results,
grouped by two-digit FoR code)
High Performing two-digit Units of Evaluation
Two-digit Fields of Research codes:
01 Mathematical Sciences
02 Physical Sciences
03 Chemical Sciences
05 Environmental Sciences
07 Agricultural and Veterinary Sciences
09 Engineering
11 Medical and Health Sciences
17 Psychology and Cognitive Sciences
18 Law and Legal Studies
21 History and Archaeology
Web: arc.gov.au I Email: Communications@arc.gov.au
Percentage of all research outputs in ERA
2010–2015
Percentage of open access research
outputs in ERA 2015 Submissions
Considering Research Impact
• Research Impact means the demonstrable
contribution that research makes to the economy,
society, culture, national security, public policy or
services, health, the environment, or quality of life,
beyond contributions to academia.
• It is included in objectives and selection criteria
• The ARC is seeking better communication about
research outcomes
Web: arc.gov.au I Email: Communications@arc.gov.au
Measuring Impact
• In 2016 the ARC will work with the higher education
research sector, industry and other
end-users of research to develop quantitative and
qualitative measures of impact and engagement
• A pilot assessment will take place in 2017
• The ARC will conduct a full assessment in 2018 as a
companion to ERA
Web: arc.gov.au I Email: Communications@arc.gov.au
Engagement and Impact Assessment
• National Innovation and Science Agenda—7 December 2015—
$1.1 billion over four years
• A range of new initiatives to support research, incentivise innovation
and entrepreneurship, reward risk taking, and promote science,
maths and computing in schools
• Includes $11.2 million over five years ($2.2 million on average) to
conduct a national assessment measuring engagement and impact
in university research
• Government invests $3.5 billion a year in university research
• ARC and Department of Education and Training to develop impact
methodology in 2016, including extensive sector consultation
Web: arc.gov.au I Email: Communications@arc.gov.au
Engagement and Impact Assessment
1. Allocation
UK REF
Web: arc.gov.au I Email: Communications@arc.gov.au
Engagement and Impact Assessment
1. Allocation
UK REF
2. Advocacy
+ Impact case studies
Web: arc.gov.au I Email: Communications@arc.gov.au
Engagement and Impact Assessment
1. Allocation
UK REF
2. Advocacy
+ Impact case studies
3. Alignment
4. Assessment
5. Analysis
6. Advancement /Improvement
Some common difficulties
• Attribution
• Time-lag
• Reference Periods
• Data collection vs reporting burden
• Finding fast feedback drivers
We need a better
and more
complete
description of
activity.
Engagement
Universities are multi-dimensional
Linkage fraction vs Discovery fraction
Mapping Engagement:
Linkage Projects vs. Discovery Projects
Engagement by cohort
Strength in Linkage and Discovery schemes
Engagement by university
Strength in Linkage and Discovery schemes
Thank you