Dr Amberyn Thomas
Manager, Scholarly Publications University of
Queensland, Australia
* Background
* The research landscape, with a particular focus on evaluation
* The UQ Library structure and services offered
* Who we are engaging with and examples of what we do
* Training and suggestions for best-practice
*Research-intensive
•
•
•
6 faculties
33 schools
9 research institutes
*47 sites
•
Hospitals, Islands & Mine
*48,800 students
•
Postgraduates – >12,580
•
•
Total >6,890
Academic > 2,880
*Staff (FTE)
*
Global university
National
university
State based
university
One of the top universities in Australia and in top
100 worldwide
* Many countries have research evaluation programs
* e.g. UK RAE since 1992 (now REF)
* Within Australia:
* ERA 2010 & ERA 2012 & will run again in 2015
* Increasing prominence of university rankings, and new rankings emerging:
“Universities are increasingly confronted by a plethora of ranking and classification
initiatives” - European University Association
* ‘Globalisation of research’ and relationship between ‘collaboration’ and
‘quality’
* New metrics – altmetrics and usage statistics
* Shift to include broader ‘impact’
Impact ‘locally’ – within UQ there
is keen focus on research quality
& impact, & projects aimed at
understanding collaboration
Deputy Vice
Chancellor
(Academic)







UQeSpace (institutional
repository)
Digitisation
Bibliometrics
Scholarly Publishing,
Open Access
Research publications
reporting and evaluation
Research Data
Management (DM plans,
repository)
Copyright
Scopus
ResearcherID
Web of
Science
Citation counts
& Publication Records
InCites
Author
Profile
UQ Staff
Data (HR)
Government reporting,
ERA, UQ data
warehouse, Academic
portfolio,
UQreSEARCHers, etc
Publication
records
Citation
Counts & Publication
Records
Benchmarking
Scopus Custom
Dataset
eSpace
repository
My UQ
eSpace
Altmetrics
Data entry
* Involved in Excellence in Research for Australia ERA 2010 & 2012
* Provide briefing notes and information about metrics-related data for e.g.
ODVC(R)
* University ranking information
* Nature and Science publishing activity
* Benchmarking data (bibliometrics)
* Produce reports on collaborative publication activity for:
* International Office, Research Office, Senior Executive
* Provide reports and metrics advice for School Reviews & major grant
applications
* Work closely with Research Information Service Librarians who work with
individual academics, research groups and Schools in the provision of
metrics services and advice
* Individual researchers and research teams
* Heads of School, Faculty, Institute Directors
* Research managers and administration staff
* Research Committees
* Senior exec: VC (President) Office, Deputy
Vice-Chancellor (Research), International office
* Commercialisation Unit – UNIQUEST at UQ
*Advise academics on what tools are available and how
useful they are for tracking research outputs & their
citations
*E.g. Web of Science, InCites, Scopus, ORCiD, Altmetrics
*Help academics identify and find relevant metrics, e.g.
h-index
*Provide context for metrics, including benchmarking and
expected citation rates for different discipline areas
where appropriate
*Advise on how to put the metrics into broader context
*Review what academics write using metrics information
and provide feedback on accuracy, relevance etc.
qual·i·ty/ˈkwälətē/
Noun
•
•
the standard of something as measured against
other things of a similar kind
the degree of excellence of something
Metrics provide supporting evidence of claims individuals
and research teams make
e.g.
Track record of individuals/teams
* Contribution to the discipline and/or significance of the
contribution
* Evidence of international/national profile
* Evidence of capacity to collaborate effectively
custom advice on research
performance & publication metrics
* Collaboration reports – country & institution
* University rankings information
* Key Performance Indicator information
* Annual report statistics
* School Review support
*
* What industries do UQ authors co-publish with?
* What industries do academics in other Australian-based
institutions co-author with?
* What industries are acknowledged as a ‘funder’ of UQ
publications?
Rank
Institution
1 BAOSTEEL CO
1 MONASH UNIV
1 SHANGHAI JIAO TONG UNIV
1 UNIV TECHNOL SYDNEY
5 BAOSTEEL STAINESS STEEL CO IRONMAKING
PLANT
5 BHP BILLITON CHINA
5 BHP BILLITON NEWCASTLE TECHNOL CTR
Web of Science
Documents
1
1
1
1
1
1
1
* Ensure adequate training opportunities and support for Librarians
to ‘learn while doing’
* In the beginning: seminars & hands-on workshops for Librarians by
specialist
* Conducted scenario-based training
* Librarians worked in groups of 2 or 3 over a 6 week period then
the groups presented their results to each other in a ½ day
workshop
* Scenario examples:
* I would like advice on how to benchmark my research performance
against others in my field within the Australian context, as well as
internationally. I am in the sciences with good WoS and Scopus
coverage.
* What is my h-index? I am in a discipline with excellent Web of Science
and Scopus coverage – I am in Biological Sciences. Why is the Scopus
value different to the WoS h-index? How does Google Scholar compare?
* I would like to demonstrate in my research grant application that my
research has found widespread application beyond my discipline. How
can I do this? I am in a discipline with good WoS and Scopus coverage.
* Worked very closely with Librarians when they first started
working with academics in this area
* Attended meetings with them
* Provided outlines of suggestions
* Proof-read written material
* Developed resources for Librarians and made these available on
the intranet
* Self-directed learning tasks
* InCites training activity
* Provided work rotations for Librarians in SPADS unit
* Conducted regular update sessions
And training is ongoing…..
The Library does not offer a standard bibliometric profiling service,
but will work with individual researchers to provide targeted,
contextualised information, and detailed advice about what tools
and methods are available for tracking research outputs and their
impact. This approach reflects the significant disciplinary differences
in the application of metrics solutions for research evaluation.
Further, the service aims to provide relevant information, which
will come about through understanding the context in which the
metrics are being used. The Research Metrics Service will support
the University’s mission and enhance world-class scholarship. In
alignment with this mission, the Library Research Metrics Service will
work with School, Institute and Centre Heads, as well as
managers of smaller research groups and centres to provide
relevant and useful metrics information. Resources permitting, the
Library will be proactive in supporting UQ’s participation in
national initiatives in the area of research evaluation, such as the
Excellence in Research for Australia, which align with UQ’s strategic
goals.
* Librarians are not ‘bibliometricians’
* For individual researchers and small research
teams, Librarians deliver a service based on
provision of advice and practical assistance
* Requires relevant resources & sophisticated tools
* Demands ongoing training and being kept up-todate with scholarly publishing trends
• Don’t reduce individual performance to a single number
• Don’t use IFs as measures of quality
• Don’t apply (hidden) “bibliometric filters” for selection
• Don’t apply arbitrary weights to co-authorship
• Don’t rank scientists according to 1 indicator
• Don’t merge incommensurable measures
• Don’t use flawed statistics
• Don’t blindly trust one-hit wonders
• Don’t compare apples and oranges
• Don’t allow deadlines and workload to compel you to drop
good practices
Wolfgang Glanzel (Leuven), Paul Wouters
(Leiden): ISSI 2013 Plenary Session
• Also individual-level bibliometrics is statistics
• Analyse collaboration profiles of researchers
• Always combine quantitative and qualitative methods
• Use citation context analysis
• Analyse subject profiles
• Make an explicit choice for oeuvre or time-window analysis
• Combine bibliometrics with career analysis
• Clean bibliographic data carefully and use external sources
• Even some “don’ts” are not taboo if properly applied
• Help users to interpret and apply your results
Wolfgang Glanzel (Leuven), Paul Wouters
(Leiden): ISSI 2013 Plenary Session
Thank-you!