Preparing for Impact
Jane Tinkler
Impact of Social Sciences Project
LSE Public Policy Group
NUI Galway, 7 June 2012
The rise of the ‘impact agenda’
• There is increased discussion of the usefulness of
‘impact’ as a measure to assess academic outcomes.
• In the UK, HEFCE has introduced an impact element to its
quality assessment process.
• All UK funders now require an impact statement to be
supplied for each funding application submitted to them.
• The ‘academic spring’ has linked impact to the opening
up of academic publications allowing research to be read
and used without needing to pay subscriptions.
• But for many academics, impact is still a confusing
concept and one that is disconnected from their
academic lives.
1. What are research impacts and how
can you measure them?
What are research impacts?
A research impact is a recorded or otherwise auditable
occasion of influence from academic research on
another actor or organization.
a. Academic impacts from research are influences
upon actors in academia or universities, e.g. as
measured by citations in other academic authors’
work.
b. External impacts are influences on actors outside
higher education, that is, in business, government
or civil society, e.g. as measured by references in
the trade press or in government documents, or by
coverage in mass media.
Tools for tracking academic impact
Tools
Pros
Cons
Bibliometric
databases such as ISI
Web of Science and
Scopus
Gives accurate
citation counts
(no duplications)

Biased towards STEM
disciplines, US and English
language outputs
 Only covers articles
Tools for tracking academic impact
Tools
Pros
Cons
‘Tweaked’ versions of
Google such as
Harzing’s Publish or
Perish
Allows
computation of
citation scores

Covers all academic
outputs that are on the web
 Easy to correct
duplications
Tools for tracking academic impact
Tools
Pros
Cons
Open search via
Google Scholar
Citations
Covers all
academic
publications

Can link to both articles
and co-authors
 Easy to use and will be
taken up quickly
2. Then how do you assess your impact
scores?
Putting your impact profile in context
A number of factors give context to your impact scores.
These can include:
• Your career position: Senior staff generally have higher
citation rates and H scores as they have published more
and have had longer for these publications to be read.
Average H scores by career stage
5.0
4.6
4.5
4.0
Avergae h-score
3.5
3.0
2.3
2.5
2.1
2.0
1.5
1.0
0.5
0.0
Lecturer
Senior Lecturer
Academic Position
Professor
Putting your impact profile in context
A number of factors give context to your impact scores.
These can include:
• Your career position: Senior staff generally have higher
citation rates and H scores as they have published more
and have had longer for these publications to be read.
• Your discipline: Disciplines vary in the outputs they
produce, their citation rates and their H scores.
Types of academic outputs by discipline
Sociology
Law
Philosophy
Academic articles
Media Studies
All book outputs
Working papers
Business and Management
Conference papers
Economics
Political Science
Psychology
0%
20%
40%
60%
80%
100%
Average H-scores by discipline and
career position
9.0
8.0
Lecturer
Average h-score
7.0
Senior
Lecturer
6.0
5.0
Professor
4.0
3.0
2.0
1.0
0.0
Discipline
Putting your impact profile in context
A number of factors give context to your impact scores.
These can include:
• Your career position: Senior staff generally have higher
citation rates and H scores as they have published more
and have had longer for these publications to be read.
• Your discipline: Disciplines vary in the outputs they
produce, their citation rates and their H scores.
• Your research focus: Government draws more on some
disciplines than others. Civil society groups are more
likely to publicly use and quote academic work.
Businesses tend to employ graduates to update their
knowledge or methods base.
Disciplinary differences in external
references
Deviation from the mean of the total
of all references
References in traditional academic locations
12
8
4
0
-4
-8
-12
-16
Pol Sci
Philo
Hist
Law
Econ
Socio
Bus &
Man
Geog
Comm &
Med Anthro Soc Pol
Psych
References in government bodies only (UK and abroad)
4
3
2
1
0
-1
-2
-3
-4
Soc Pol
Psych
Geog
Socio
Econ
Law
Pol Sci
Bus &
Man
Anthro
Comm &
Med
Hist
Philo
3. Planning for Impact
Academic communication is changing
Academic communication involves:
• Journal articles, conference papers, books and reviews.
• Journal articles and books are read by some in your field,
but don’t often break into other disciplines and rarely
picked up by the media.
• Outputs are often fairly long and in language that is
sometimes meaningful only to other academics.
BUT
• Academics are observers who need to communicate
their observations to the world (in a timely fashion).
• Much of our knowledge and input goes unapplied
because of very long time-lines for outputs, and lack of
adaptation or translation.
Seven steps to creating impact
• Step 1: Think about how your research and what types of
outputs you produce.
Co-authorship and citations
Co-authorship and
Number of Outputs
Most outputs in our
dataset were single
authored, but more cites
went to outputs that had
at least one other author
1000
800
600
400
200
Co-authorship and Citations
0
0
1
2
3
4
5
Number of Co-authors
6 or 7000
more
6000
Citations received
Number of Outputs
1200
5000
4000
3000
2000
1000
0
0
1
2
3
4
Number of Co-authors
5
6 or
more
Seven steps to creating impact
• Step 1: Think about how your research and what types of
outputs you produce.
• Step 2: Pick as distinctive a version of your author name
as possible and stick with it.
• Step 3: Write informative article titles, abstracts and
book blurbs.
• Step 4: Build communication and dissemination plans
into research projects early on.
Seven steps to creating impact
• Step 5: Make full use of your university’s resources
(like online depositories, Expert directories, knowledge
transfer schemes) as well as using public resources like
creating a profile in Google Scholar Citations.
• Step 6: Ensure you put a version of all publications on
the open web and use social media to raise the profile
of your research, e.g. write blogs, write online book
reviews, tweet.
A team from the World Bank examined the influence
of economic blogs on download figures for articles
A team from NCRM compared the effect on a paper’s
downloads via twitter and other routes
Seven steps to creating impact
• Step 5: Create a public profile on your university site
and on Google Scholar Citations.
• Step 6: Ensure you put a version of all publications on
the open web and use social media to raise the profile
of your research, e.g. write blogs, write online book
reviews, tweet.
• Step 7: Consider working with intermediate
organisations where possible to help disseminate your
work and create impact e.g. think tanks, community
groups.
What constrains impacts?
Higher Education
Institution
Private / public / third
sector organisation
 Lack of time
 Bureaucracy and inflexibility of
HEI administration
 Difficulties in identifying partners
 Insufficient rewards and lack of
awareness of the benefits from
the interactions
 Lack of understanding by
academics of the process
 Capacity and capability of the KE
system still developing / evolving
 Lack of resources within
external organisations to fund
the KE engagement
 Insufficient benefits from the
interaction
 Lack of interest by external
organisations and lack of
demand for KE
 Intellectual property
agreements as a barrier to
some, albeit minority of, KE
engagement
Source: PACEC/CBR Survey of Academics (2008); PACEC/CBR Survey of
Enterprise Offices (2010); CBR Survey of Enterprises (2008)
4. What picture of external impact has
our research found?
Mapping distribution of impacts
We used Google to search 264 social science academics and
recorded where we found references to them or their
research.
Academic research and
engagement
Civil society and
third sector
(7%)
Govt &
policy
(5%)
Digital aggregators
(4%)
Academic
publishers
and journals
(20%)
Digital
research
databases
Media and
press
(5%)
Individs
(4%)
Academic
assocs.
and
societies
(7%)
All libraries
(14%)
University departments (20%)
Private
sector
(3%)
Independent
think tanks
(4%)
Univ.
centres
and
instits.
(7%)
66 per cent of references are
international
Where we found social science
references in UK government
Research Councils
UK Parliamentary
Local Govt
Central Govt Depts
NDPBs
Other
0
5
10
15
20
25
30
35
% of total UK government references
N = 325. ‘Central
Government Departments’ includes the Home Office, the
Department for Work and Pensions, and the Department for International Development,
among others. ‘NDPB’ refers to Non-departmental Public Bodies (excluding Research
Councils) and includes the National Archives.
Notes:
Where we found social science
references in business
Major corporations
Business type
Associations
Mediators
SMEs
0
5
10
15
20
25
% of total private sector references
30
35
For more see:
Maximising the Impacts of your Research: A handbook for social
scientists
Using Twitter in University Research, Teaching and
Impact Activities: A guide for academics and
researchers
Freely available to download from the
Impact of Social Sciences blog:
http://blogs.lse.ac.uk/impactofsocialsciences/
Email: impactofsocialsciences@lse.ac.uk
Twitter: @lseimpactblog
Facebook: Impact of Social Sciences