A ‘how to’ guide to measuring your
own academic and external impacts
Patrick Dunleavy and Jane Tinkler
LSE Public Policy Group
Investigating Academic Impacts conference
Monday 13 June 2011
Structure of this presentation
1.
2.
3.
4.
The ‘impacts agenda’ and PPG’s ‘evidence base’
Academic citations:
 Where to start
 How well cited are you?
 Tips for increasing academic citations
External impacts:
 Key factors shaping external impact
 Differences across roles and disciplines
 Tips for increasing external impacts
Conclusions
1. The ‘impacts agenda’




There is a significant imbalance in funding
for social sciences compared to STEM
subjects
Plus cuts to funding for the university
sector as a whole
It is important to be able to show the value
of academic research in general
But also we all want our work to be seen,
read, used and have impact
1b A word on PPG’s evidence base




Compiled a dataset of 240 academics
across 10 social science disciplines from
across the UK
Looked at their academic citations and
their external impacts
This research forms the basis for our
conclusions. New findings will be updated
on our blog and in next iterations of the
handbook
There is no magic solution but there are a
number of practical things that you can do
now
2a. Academic citations: Where to start?
Tools
Pros
Cons
Bibliometric
Gives accurate
 Biased towards STEM
databases such as ISI citation counts
disciplines, US and English
Web of Science and
(no duplications) language outputs
Scopus
 Only covers articles
Open search via
Google Books and
Google Scholar
Covers all
academic
publications
‘Tweaked’ versions of Allows
Google such as
computation of
Harzing’s Publish or citation scores
Perish
Includes duplications
and mistakes
 Citations can become
blurred and over-inclusive

No cons we can see (so
far!)
The inclusiveness of the ISI database for items
submitted to the UK’s Research Assessment Exercise,
2001
Clinical medicine
Discipline group
Biological sciences
Physical sciences
Health sciences
Mathematics
Health allied
Engineering and Computer Sciences
Social sciences
0 10 20 30 40 50 60 70 80 90 10
0
Percentage of research items submitted in the 2001 RAE that were
included in the ISI database
2b. Academic citations:
How well cited are you?
Simple indicators can be used:
 Your total number of publications
 Your total number of citations (a better
representation than citations per output)
 Your H-score (the number of outputs each
being cited that number of times), Age
Weighted H-score or G Index (takes into
account highly cited top publications)
How the H-score and G-score works
g index = average (mean) citations
of items above h line only
parity line
2b. Academic citations:
How well cited are you?
Then take into account:
 Your career position (early-year, senior
lecturer, professor)
 Your discipline
 How you work (single vs multiple author,
single vs multi-discipline, applied vs
theoretical research, hub vs authority
referencing)
Average H-scores by discipline and
career position
8.00
7.00
Average h-score
6.00
5.00
4.00
3.00
2.00
1.00
0.00
Economics
Geography
Lecturer
Sociology
Senior Lecturer
Political Science
Professor
Law
2c. Academic citations:
Tips for increasing citations






Pick as distinctive a version of your author
name as possible and stick with it
Write informative article titles, abstracts and
book blurbs
Work with colleagues to produce multiauthored outputs
Consider cross-disciplinary research projects
Build communication and dissemination plans
into research projects early on
Always put a version of any output on the
open web
3a. External impacts: Key factors shaping
the external influence of academics
6b. Public, media
reputation
7. Experience
6a. ‘Insider’,
elite
reputation
6. External
reputation
8. Track
record
1. Academic
credibility
5. Interaction
expertise
4. Personal
communication
capacity
2. Dispositional and
sub-field constraints
3. Networking
skills
Key: Low
Medium High
3b. External impacts:
Differences across roles and disciplines



Positions: early-years researcher, senior
lecturer, professor
Academic roles: research, teaching,
academic citizenship, academic
management, dissemination
Disciplines: subject areas are more linked
in to particular external groups
External impacts: Differences by discipline
Mentions to Economists
Mentions to Political Scientists
Think Tank
Think Tank
30
35
30
Group or Indiv
25
25
Government
Group or Indiv
Government
20
20
15
15
10
10
5
5
0
Society
Business
Society
Press
0
Business
Library
Library
Other/NA
Other/NA
Mentions to Geographers
Think Tank
30
25
Group or Indiv
Government
20
15
10
5
Society
0
Business
Press
Library
Other/NA
Press
3c. Tips for increasing external impacts
for academics




Most importantly, create an ‘impact file’ to collect
information on all your external interactions
Consider alternative methods of disseminating
research outputs that are tailored to particular
audience groups
Research mediators such as think tanks or
community groups are a good way to link into
networks of interest
Use all available dissemination resources e.g.
online depositories, seminar series, multi-author
blogs, knowledge transfer schemes
3d. Tips for increasing impacts scores for
universities and departments






Provide an overall steer on the value of
dissemination and impact for all academic staff
Incentivise this through promotion and
performance processes
Factor dissemination and impact into calculations
of academic workload and time burdens
Re-evaluate event /conference programmes
Host online depositories or other dissemination
opportunities such as blogs
Facilitate collaboration and linking to dedicated
expert teams/consultancies
4. Conclusions



Maximizing both academic and external
impacts helps promotion and career
fulfillment – and via the REF it may bring
additional money for your university
There are resources available to help, such
as the HEIF fund - £600m shared across 98
universities for the 2011-2015 period
For the REF in 2014, 20 per cent of the
funding will come from the external impact
assessment, and a 4* impact case study may
now bring in as much research grant as 13
publications rated at 4*
For more details see:
Maximising the Impacts of your Work handbook
Impact of Social Science blog:
http://blogs.lse.ac.uk/impactofsocialsciences/
Email: impactofsocialsciences@lse.ac.uk
Twitter: @lseimpactblog
Facebook: Impact of Social Sciences