Dr Amberyn Thomas Manager, Scholarly Publications University of Queensland, Australia * Background * The research landscape, with a particular focus on evaluation * The UQ Library structure and services offered * Who we are engaging with and examples of what we do * Training and suggestions for best-practice *Research-intensive • • • 6 faculties 33 schools 9 research institutes *47 sites • Hospitals, Islands & Mine *48,800 students • Postgraduates – >12,580 • • Total >6,890 Academic > 2,880 *Staff (FTE) * Global university National university State based university One of the top universities in Australia and in top 100 worldwide * Many countries have research evaluation programs * e.g. UK RAE since 1992 (now REF) * Within Australia: * ERA 2010 & ERA 2012 & will run again in 2015 * Increasing prominence of university rankings, and new rankings emerging: “Universities are increasingly confronted by a plethora of ranking and classification initiatives” - European University Association * ‘Globalisation of research’ and relationship between ‘collaboration’ and ‘quality’ * New metrics – altmetrics and usage statistics * Shift to include broader ‘impact’ Impact ‘locally’ – within UQ there is keen focus on research quality & impact, & projects aimed at understanding collaboration Deputy Vice Chancellor (Academic) UQeSpace (institutional repository) Digitisation Bibliometrics Scholarly Publishing, Open Access Research publications reporting and evaluation Research Data Management (DM plans, repository) Copyright Scopus ResearcherID Web of Science Citation counts & Publication Records InCites Author Profile UQ Staff Data (HR) Government reporting, ERA, UQ data warehouse, Academic portfolio, UQreSEARCHers, etc Publication records Citation Counts & Publication Records Benchmarking Scopus Custom Dataset eSpace repository My UQ eSpace Altmetrics Data entry * Involved in Excellence in Research for Australia ERA 2010 & 2012 * Provide briefing notes and information about metrics-related data for e.g. ODVC(R) * University ranking information * Nature and Science publishing activity * Benchmarking data (bibliometrics) * Produce reports on collaborative publication activity for: * International Office, Research Office, Senior Executive * Provide reports and metrics advice for School Reviews & major grant applications * Work closely with Research Information Service Librarians who work with individual academics, research groups and Schools in the provision of metrics services and advice * Individual researchers and research teams * Heads of School, Faculty, Institute Directors * Research managers and administration staff * Research Committees * Senior exec: VC (President) Office, Deputy Vice-Chancellor (Research), International office * Commercialisation Unit – UNIQUEST at UQ *Advise academics on what tools are available and how useful they are for tracking research outputs & their citations *E.g. Web of Science, InCites, Scopus, ORCiD, Altmetrics *Help academics identify and find relevant metrics, e.g. h-index *Provide context for metrics, including benchmarking and expected citation rates for different discipline areas where appropriate *Advise on how to put the metrics into broader context *Review what academics write using metrics information and provide feedback on accuracy, relevance etc. qual·i·ty/ˈkwälətē/ Noun • • the standard of something as measured against other things of a similar kind the degree of excellence of something Metrics provide supporting evidence of claims individuals and research teams make e.g. Track record of individuals/teams * Contribution to the discipline and/or significance of the contribution * Evidence of international/national profile * Evidence of capacity to collaborate effectively custom advice on research performance & publication metrics * Collaboration reports – country & institution * University rankings information * Key Performance Indicator information * Annual report statistics * School Review support * * What industries do UQ authors co-publish with? * What industries do academics in other Australian-based institutions co-author with? * What industries are acknowledged as a ‘funder’ of UQ publications? Rank Institution 1 BAOSTEEL CO 1 MONASH UNIV 1 SHANGHAI JIAO TONG UNIV 1 UNIV TECHNOL SYDNEY 5 BAOSTEEL STAINESS STEEL CO IRONMAKING PLANT 5 BHP BILLITON CHINA 5 BHP BILLITON NEWCASTLE TECHNOL CTR Web of Science Documents 1 1 1 1 1 1 1 * Ensure adequate training opportunities and support for Librarians to ‘learn while doing’ * In the beginning: seminars & hands-on workshops for Librarians by specialist * Conducted scenario-based training * Librarians worked in groups of 2 or 3 over a 6 week period then the groups presented their results to each other in a ½ day workshop * Scenario examples: * I would like advice on how to benchmark my research performance against others in my field within the Australian context, as well as internationally. I am in the sciences with good WoS and Scopus coverage. * What is my h-index? I am in a discipline with excellent Web of Science and Scopus coverage – I am in Biological Sciences. Why is the Scopus value different to the WoS h-index? How does Google Scholar compare? * I would like to demonstrate in my research grant application that my research has found widespread application beyond my discipline. How can I do this? I am in a discipline with good WoS and Scopus coverage. * Worked very closely with Librarians when they first started working with academics in this area * Attended meetings with them * Provided outlines of suggestions * Proof-read written material * Developed resources for Librarians and made these available on the intranet * Self-directed learning tasks * InCites training activity * Provided work rotations for Librarians in SPADS unit * Conducted regular update sessions And training is ongoing….. The Library does not offer a standard bibliometric profiling service, but will work with individual researchers to provide targeted, contextualised information, and detailed advice about what tools and methods are available for tracking research outputs and their impact. This approach reflects the significant disciplinary differences in the application of metrics solutions for research evaluation. Further, the service aims to provide relevant information, which will come about through understanding the context in which the metrics are being used. The Research Metrics Service will support the University’s mission and enhance world-class scholarship. In alignment with this mission, the Library Research Metrics Service will work with School, Institute and Centre Heads, as well as managers of smaller research groups and centres to provide relevant and useful metrics information. Resources permitting, the Library will be proactive in supporting UQ’s participation in national initiatives in the area of research evaluation, such as the Excellence in Research for Australia, which align with UQ’s strategic goals. * Librarians are not ‘bibliometricians’ * For individual researchers and small research teams, Librarians deliver a service based on provision of advice and practical assistance * Requires relevant resources & sophisticated tools * Demands ongoing training and being kept up-todate with scholarly publishing trends • Don’t reduce individual performance to a single number • Don’t use IFs as measures of quality • Don’t apply (hidden) “bibliometric filters” for selection • Don’t apply arbitrary weights to co-authorship • Don’t rank scientists according to 1 indicator • Don’t merge incommensurable measures • Don’t use flawed statistics • Don’t blindly trust one-hit wonders • Don’t compare apples and oranges • Don’t allow deadlines and workload to compel you to drop good practices Wolfgang Glanzel (Leuven), Paul Wouters (Leiden): ISSI 2013 Plenary Session • Also individual-level bibliometrics is statistics • Analyse collaboration profiles of researchers • Always combine quantitative and qualitative methods • Use citation context analysis • Analyse subject profiles • Make an explicit choice for oeuvre or time-window analysis • Combine bibliometrics with career analysis • Clean bibliographic data carefully and use external sources • Even some “don’ts” are not taboo if properly applied • Help users to interpret and apply your results Wolfgang Glanzel (Leuven), Paul Wouters (Leiden): ISSI 2013 Plenary Session Thank-you!