Excellence in Research for
Australia 2015 Update
Ms Leanne Harvey
Executive General Manager
Introduction
There have been several changes to the Submission Guidelines for ERA
2015. However, the ERA objectives are unchanged:
1. Establish an evaluation framework that gives government, industry, business
and the wider community assurance of the excellence of research conducted
in Australian higher education institutions
2. Provide a national stocktake of discipline level areas of research strength and
areas where there is opportunity for development in Australian higher
education institutions
3. Identify excellence across the full spectrum of research performance
4. Identify emerging research areas and opportunities for further
development and
5. Allow for comparisons of research in Australia, nationally and internationally,
for all discipline areas.
2
ERA 2015 Reference Periods
Data type
Reference period
Years
Research outputs
1 Jan 2008-31 Dec 2013
6
Research income
1 Jan 2011-31 Dec 2013
3
Applied measures
1 Jan 2011-31 Dec 2013
3
Esteem measures
1 Jan 2011-31 Dec 2013
3
Staff census date: 31 March 2014
We: arc.gov.au
Email: era@arc.gov.au
3
High Level Changes for ERA 2015
•
•
•
•
•
•
•
•
•
•
•
Gender Data
Open Access
Submission Stage 0
Treatment of Revisions, Reprints and Multiple Editions
Digital Storage of Research Outputs Nominated for Peer
Review
ERA Submission Publisher ID
ERA Submission Conference ID
Publication Association for Staff Employed < 0.4 FTE
New Category of Non-Traditional Research Outputs
Nomination of Research Outputs for Peer Review
Peer Review Reporting
4
Preparation for ERA 2015
• ERA team – university visits – 38 so far!
• New ERA 2015 Journal list contains 24,028 journals – 3,123
journals had FoR changes and 1,589 new journals were added
• The Journal and Conference lists will be released at the end of
ERA and will include a full list of journals where ERA-eligible
publications have been submitted
• Stage 0 Submissions open soon – 19 January 2015
• You are already working on it
We are here to help - ERA Helpdesk
5
Preparation for ERA 2015
Submission top tips
• Know the submission stages and timetable
• Use Submission Stage 0
• Develop your ERA Peer Review samples
• Develop your Explanatory Statements
• Set up and internally test repositories
• Think carefully about reassignment!
6
ERA 2015 Submission Timetable
Phase
Activity
Start Date
Deadline
Responsible
Stage 0
19 January 2015
19 February 2015
Institutions
Stage 1
23 February 2015
13 March 2015
Institutions
Stage 2
16 March 2015
7 April 2015
ARC, with
Institutions
Stage 3
8 April 2015
13 April 2015
Institutions
Stage 4
14 April 2015
20 April 2015
Institutions
Submission
7
Submission Stage 0
• Upload trial submissions to test XML data structure
• Opens 19 January 2015 to 19 February 2015
• All business rules available - except BR024 and BR117 relating to EID
tagging
• New report - 30% ERA peer review sample
• Units of evaluation will not be created
• Submissions uploaded will not carry forward to Stage 1
8
Submission Upload Stats ERA 2012
Institution
Upload Attempts
Highest
132
2nd Highest
111
3rd Highest
105
4th Highest
97
5th Highest
92
5th Lowest
18
4th Lowest
18
3rd Lowest
16
2nd Lowest
14
Lowest
13
Grand Total
1924
9
Nomination of Outputs for ERA Peer Review
In the peer review disciplines, universities are required to provide a 30% sample of the
research outputs in the 4 digit unit of evaluation for ERA peer review.
What is a ‘representative sample’?
• Should be drawn from a representative sample of the eligible researchers
• Must include 30% of apportioned outputs (rounded up to the nearest integer) for each
output type:
• Books
• Book Chapters
• Journal Articles
• Conference Papers
• Non Traditional Research Outputs
• No minimum threshold - if only one NTRO is submitted that NTRO must be nominated
for peer review.
10
Nomination of Outputs for ERA Peer Review cont.
Peer Review Samples for Each Research Output Type – University X, FoR 1603 (Demography)
Book
Journal Article
Conference
Paper
Non-Traditional
Research Output
Total Research
Outputs
Research volume
(apportioned value of
outputs)
Peer Review sample
(number of outputs)
10.00
26.66
31.00
0.7
68.36
3
8
10
1
22
Peer review sample
(% required of research
volume)
30%
30% (rounded
up to the next
integer)
30% (rounded up 30% (rounded up
to the next
to the next
integer)
integer)
Greater than
30%
11
Nomination of Outputs for ERA Peer Review cont.
Finally, think of the peer reviewers when getting your sample
together.
My picks from ERA 2012 were:
•
“10 Stiff Undies”
•
“Beyond 'Do No Ham'……”
•
“Ooga Booga”
12
Explanatory Statements
Do:
• Provide an Explanatory Statement for all two-digit units of evaluation likely to be evaluated
• Provide contextual information about publication profile, research environment and capacity,
collaboration, range of research activity, and other information such as awards/prizes
• Explain perceived anomalies or unusual patterns
• Limit your statement to 10 000 characters including spaces
Don’t:
• Make claims not supported by the submission data
• Refer to individual researchers
• Refer to previous ERA ratings
• Refer to other university ranking systems and results
• Include additional citation information not included in ERA
• Simply repeat information that is already included in the submission
• Include reference to information outside the reference period (including future directions)
• Include embedded links
13
Explanatory Statements cont.
I do the checks on these and whilst I love a good laugh - proofread your
statements and perhaps just perhaps ………. think twice about comments like:
(note these are live ones from ERA 2012)
•
A hallmark of our approach is productive methodological eclecticism
•
You have to understand that we, at this institution, suffer from small discipline
syndrome
•
Our research office is staffed by idiots and they forgot to claim our research income but
you should add $X million into our discipline
14
Repository Testing Instructions
• Guide to testing for
Repository Managers
• Will be distributed this
week through ELOs
15
REC Member and Peer Reviewer Recruitment
Research Evaluation Committee (REC) Membership
• 8 REC Chairs announced in November - bring considerable experience and ability to
ensure ERA evaluations are of the highest standard
• The ARC is in the process of contacting other selected members of the RECs
• The ARC will announce the full list of RECs members in the coming months after all
contracts have been finalised
• ELOs will be advised
Peer Reviewers
• The call for Peer Reviewer nominations closed on 26 November 2014
• Nearly 1000 nominations – even better than 2012!
• Unsuccessful REC nominees may be contacted to participate in the peer review process
16
ERA-HERDC Alignment
Background
• PhillipsKPA Review of Reporting Requirements for Universities
• Department of Education and the ARC working together
• A single research data collection
Aim
• A more efficient streamlined process for the collection of data to inform RBG and evaluate
excellence
Potential Efficiencies
• University admin staff and IT systems only need to cater for one data set
• Reduces opportunities for error and misunderstanding - enhancing integrity, transparency and
utility of data
• Auditing requirements in relation to the data will be streamlined
Sector Consultation
• Discussion paper including possible options will be sent to universities soon
17
Questions?
18