Innovation Measurement
Keith Smith
Imperial College London/TIK Oslo
Why do we need data?
• Economy-wide data enables a structural,
generalisable view to emerge
• It allows us to explore the properties of a
system as a whole
• It helps us to identify where the really relevant
questions are
The background issues
Historically, 3 sources of data:
• R&D
• Patents
• Bibliometric
Each has more or less serious problems as
innovation indicators
Problems with existing indicators
• All have problems with their conceptual and
definitional bases
• Two are by-products of legal or institutional
processes – patent law or academic publishing
conventions
• None focus directly on innovation
Research and Development Data
• Collected by survey, procedures formalised in
OECD ‘Frascati Manual’ (1968)
• Collects data on expenditure on R&D,
personnel employed (in FTEs), types of
research (basic, strategic, applied,
experimental), object (by field)
• Monitored by OECD NESTI working party
R&D Indicators
• The most common indicator: ‘R&D Intensity’
• R&D Intensity = R&D/GDP or R&D/GVA ratio
• Countries and firms can be ranked using this
ratio
• It is often used as a policy target (Norway –
target to reach OECD average for R&D/GDP;
EU target ‘to reach 3%’)
Problems with R&D intensity indicator
• The overall indicator reflects not only R&D effort
but also the industrial structure of the country
• If the country is heavily based on low R&D
industries, then the aggregate indicator will be
low even if the country is relatively R&D intensive
– so the aggregate intensity indicator is
misleading as in terms of country efforts (Norway
has low R&D/GDP even though it is relatively high
in many industries)
R&D and high tech sectors
• The OECD uses R&D to distinguish between technology
intensity of industries
• High tech= >4% R&D/GVA ratio
• Medium tech = between 1 and 4 %
• Low tech = <1%
But this only indicates R&D performance, it does not
reflect use of science, non-R&D inputs, technology flows
etc. By this criterion food is a low tech sector, when
actually it is strongly science using.
Patents
• A patent is a grant of monopoly use of a
discovery, usually for a period of 17 years
• The discovery must be an advance in the state
of the art, and non-obvious
• Problems: patents are only rarely taken into
use. Their economic value usually varies
enormously. Very few firms patent. Research
shows that patenting is not a strong method
of appropriation.
Bibliometric data
• Data on scientific publication and citations
(publications from ‘World of Science’, citations
from Science Citation Index)
• Widely collected and widely available by field
• ‘High Impact’ publications are in the top 1
percent of highly cited publications
• Can map relative national performance, filed
changes, international collaboration
• Can indicate surprising changes in world patterns
Innovation indicators
• Emerged in 1980s as researcher-driven
exercises in France, Germany, USA, Italy,
Scandinavia
• Development of OECD ‘Innovation manual’
(the ‘Oslo Manual’) in early 1990s
• First Community Innovation Survey 1992
The Community Innovation Survey
Covers:
• Direct outputs of innovation – sales from new
and technologically changed products
• Inputs – R&D, design, marketing, training,
acquisition of licences etc
• Collaboration – partners and locations
• Sources of information
• Incentives and Obstacles
CIS
• Now implemented six times, currently every
two years
• Funded and overseen by European
Commission (Eurostat in Luxembourg)
• Frequently revised by R&D and Innovation
working party – covers sampling and
collection methodologies
• Also collected in Canada, Australia, China,
India, Brazil, Russia, South Africa.
Main CIS results – what did we learn
• Innovation drives growth – the CDM model
• Much weaker role of R&D than expected
• Pervasiveness of innovation – especially in
‘low tech’ sectors
• Asymmetry in innovation performance
• Central role of collaboration
• Characteristics of highly innovating firms
(distributed across all sectors)
Publications using CIS
Academic papers (in English) using Community Innovation Survey
data (1994-2011)
427 100
450
366
Number of studies
400
Cumulative…
350
316
300
261
250
200
140
150
81
100
50
4
9
165
182
211
100
34 50
14 20 27
0
90
80
70
60
50
40
30
20
10
0
1994 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011
Publication year
Publications and versions of CIS
Use of each CIS version over time
40
CIS 1
CIS 2
CIS 3
CIS 4
CIS 2006
CIS 2008
35
Number of academic papers
30
25
20
15
10
5
0
1994
1996
1997
1998
1999
2000
2001
2002
2003
2004
Publication year
2005
2006
2007
2008
2009
2010
2011