Snowball Metrics
An institutional perspective
Anna Clements
4/8/2015
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds, Queen’s University Belfast,
University of St. Andrews, Elsevier
1
The origins of Snowball Metrics
Sentiments among English research institutions were the prelude to this partnership
•Growing recognition of the value of data/metrics to inform and monitor institutions’ research strategies
•Dissatisfaction with available tools: multiple custom implementations, incompatibility of data systems
•Frustration over the lack of standard metrics available to give sensible, relevant measurements
•Frequent data requests from external bodies put pressure on institutions to collect and return data on
aspects of performance that are not necessarily best suited to the institutions themselves
Imperial College London and Elsevier conducted a study of research information
management in England funded by the Joint Information Systems Committee (JISC)
Highlighted recommendations from the study
• Institutions and funders should work more collaboratively, and develop stronger
relationships with suppliers
• An agreed national framework for data and metric standards is needed
• Suppliers should participate in the development of data and metric standards
... which led to Snowball Metrics
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier
Defining Snowball Metrics
A subset of participants initiated a second phase
of work that would address and build on these
recommendations
Project Partners
• Vision: Institutions agree on metrics and methodologies as
standards to support strategic decision making by enabling
benchmarking across institutions globally
• Approach: Eight UK higher educations institutions working with
a supplier, Elsevier, partnered to reach consensus on a set of
metrics. Snowball Metrics reflect universities’ view of the world,
not Elsevier’s. Elsevier played a supporting role to help higher
education institutions articulate their needs, and to test the
feasibility of addressing them
• Outcome: Agreed and tested Snowball Metrics methodologies
have been shared freely in the Recipe Book, so that any
organization can adopt the framework
• Next steps: All project partners are committed to facilitating a
global definition of these metrics as standards by incorporating
other existing standards into the “recipes”, and working with all
suppliers to implement these metrics
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier
Summarising
• Owned by research organisations – not imposed
by funders or third parties
• Metric definitions ‘recipes’ available to all
• Using/building on existing standards – CERIF;
HESA cost centres
• Minimise burden
4/8/2015
Maximise usefulness
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds,
Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)
4
Snowball Metrics Recipe Book
Launch
Dr Lisa Colledge
Elsevier
l.colledge@elsevier.com
4/8/2015
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds, Queen’s University Belfast,
University of St. Andrews, Elsevier
5
Snowball Metrics Landscape
• Metrics are sliced at the institutional and discipline / funder type levels
• Metrics need a combination of institutional, proprietary, and public data
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial College London, University of Bristol, University of Leeds, Queen’s University Belfast, University of St. Andrews, Elsevier
Global standard metrics need to be “SMART”
•
•
•
•
•
Specific and Scalable
Manageable
Agreed
Realistic across the sector
Timebound…
4/8/2015
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds,
Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)
7
How did we make Snowball Metrics SMART?
• Data Experts from the institutions agree
theoretical consensus
• Elsevier developed a feasibility pilot
– Make the calculations with actual data
– Allow the project partners to view their metrics over
time and gain context around their performance
4/8/2015
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds,
Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)
8
Towards global standards
Share
Visibility
Accessibility
Feedback
4/8/2015
Recipe Book is free of charge
• www.snowballmetrics.com
Webinar
Regional workshops UK
Express metrics in CERIF
Incorporate in suppliers’ tools
Fit of recipes to national ingredients?
What would help you to adopt?
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds,
Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)
9
Thank you for your attention!
4/8/2015
Snowball Metrics Project Partners
University of Oxford, University College London, University of Cambridge, Imperial Colledge London, University of Bristol, University of Leeds,
Queen’s University Belfast, University of St. Andrews, Elsevier *Ordered according to productivity 2011 (data source: Scopus)
10