Conceptual and Empirical Challenges of Evaluating the Effectiveness of
Innovation Policies with ‘Behavioural Additionality’
(the Case of IWT R&D Subsidies)
Jan Larosse
IWT-Flanders
Abstract
Additionality is a key concept for the evaluation of the ‘effectiveness’ of policy instruments
for stimulating R&D and innovation.
It is historically linked with the conceptual framework of ‘market failure’ as a rationale for
government intervention in R&D or knowledge creation and diffusion in general. It has a
theoretical foundation in ‘welfare economics’: when the ‘optimal’ level of R&D investments
is not attained, public incentives can ‘make a difference’.
Because the social optimal level of R&D cannot be calculated, evaluations try to measure if
government incentives (subsidies) are or are not crowding out private investments in R&D.
This is done without referring to an optimal level, but taking the existing market conditions as
given. These evaluations are rather narrow: additionality equals ‘not substituting’, a negative
motivation, and are only interested in the financial effects.
But the concept of additionality has more potential for policy evaluation than that, as it is
linked with the idea of ‘compensating’ the spillovers of knowledge creation, generated by the
special character of knowledge as semi-public good. Knowledge can be used without wearing
out and thus contributes to cumulative growth. Dynamic spillovers are in fact the source of
productivity gains that are a multiple of the private productivity gains for the private investor.
This means that government has an interest to stimulate private R&D because it generates
social benefits that go beyond the simple under-investment hypothesis. Additionality has a
much broader meaning in this context.
Here we have to extend the reference framework of additionality to the conceptual model of
the innovation system, because this is grounded in the understanding of innovation as an
interactive and cumulative process. The existence of ‘system failures’ provides a rationale for
policy to ‘make a difference’ in unleashing the synergy potential of innovation systems that
are not performing well because of unbalances, lock-ins, weak connectivity and other
systemic dysfunctions that are related to the interaction pattern of actors.
Therefore we can use the concept of additionality as a ‘bridging concept’ between the older
and the newer models of innovation and government intervention. It has to be extended from
the linear model in which ‘input’ and ‘output’ additionality are supposed to be closely
correlated to a process oriented non-linear model in which ‘behavioural additionality’ is the
key of government policy. Learning takes place in all circumstances, but it is now up to
policy evaluation to continuously improve the leverage of policy instruments to increase the
learning capabilities on actor level and institutional level and the knowledge distribution
capacity of the system. Assessment of the effectiveness of individual instruments (as
subsidies) therefore is part of managing innovation systems. Evaluating instruments from the
perspective of their (comparative) behavioural additionality is a step towards new systembased evaluation practices, but much work needs still to be done. This paper discusses the
challenges.
Behavioural Additionality
Jan Larosse
1
16/02/16
Conceptual and Empirical Challenges of Evaluating the Effectiveness of
Innovation Policies with ‘Behavioural Additionality’
(the Case of IWT R&D Subsidies)
Contents
1. Introduction
2. Evaluation and ‘additionality’ in Flanders
2.1. Actual practices of evaluation
2.2. The new perspective of government on innovation
3. About the concept of Behavioural Additionality
3.1. Additionality
3.2. Behaviour
3.3. Kinds of additionality
4. About the measurement of behavioural additionality
5. Policy learning
6. Organisation of the work programme in Flanders
Jan Larosse is Scientific Advisor at IWT Flanders, the Innovation Agency of the Flemish
government (Belgium). He coordinates the IWT Observatory, the analytical unit of IWT. His
work relates to innovation monitoring and innovation studies covering themes such as the
knowledge economy, cluster analysis and innovation systems, as well as policy evaluation
and additionality.
Behavioural Additionality
Jan Larosse
2
16/02/16
1. Introduction
The 3% target for R&D investments, the increasing budgets for innovation policies in general,
and – foremost - the emergence of more integrated and encompassing kinds of innovation
policies, call for increased and renewed forms of evaluation of the effectiveness of these
governmental efforts.
The introduction of the concept of ‘behavioural additionality’ in recent years is an attempt to
enlarge the traditional evaluation methods based on ‘input’ and ‘output’ additionality and link
them with the policy framework of the national innovation system.
The need for of such a concept is demonstrated by the fact that it expresses a ‘catching-up’ of
policy and evaluation theory on already widely applied practices of policy makers to
explicitly target behavioural changes in the design of policy instruments. Most notorious
examples are programmes aiming at fostering co-operation between the actors in the
innovation system in order to increase the productivity of the knowledge investments through
spillovers.
The inability of present monitoring and evaluation methods to adequately assess the
effectiveness of the new system oriented policies are an important bottleneck for their
generalisation and therefore for performance improvement of innovation systems. System
policies capitalise on the synergistic character of the innovation process, while traditional
policies (and evaluation methods) only perceive sectoral policy actions towards individual
agents on project level.
Therefore increased efforts to strengthen the capabilities for evaluation are needed. Because
of the recent and still immature nature of the new approach on ‘behavioural additionality’
efforts for conceptual clarification and for standardisation of empirical research methods can
contribute to form an international community of practice that may foster the capacity for
international policy learning. The work of the OECD to establish a framework for research
and for exchange of good practices on this subject is of great importance for national
innovation policies.
The aim of this document is to contribute to the clarification of the research agenda, based on
the experience of IWT Flanders in setting-up an evaluation method for its R&D-subsidies.
The following chapter starts from the experience of evaluation of bottom-up innovation
stimulation in Flanders. Additionality is embodied in several specific selection requirements.
This practice expresses a growing understanding of the complexity of innovation processes
but evaluation of the effectiveness of these schemes by their stated objectives cannot cover
the new systemic perspective of the role of government. Behavioural additionality appears as
a ‘bridging concept’ in the evolution towards third generation innovation policies that takes
account of the interaction effects of public and private strategies.
The next chapters discuss the opportunities and limits of this new approach on the level of the
concept and its measurement. The main message of introducing behavioural additionality is
the focus on learning, including policy learning, to enhance innovation performance.
The last chapter briefly describes the complementary components of the comprehensive
evaluation strategy that is needed to implement this evaluation perspective and subsequent
new role for public support agencies in the innovation system.
2. Evaluation and ‘additionality’ in Flanders
2.1. Actual practices of evaluation
IWT is the agency of the Flemish government for the implementation of the Flemish
innovation policy but has also important tasks in policy preparation. It was established in
1992 to give shape to the new competences in science and technology that were transferred
from the federal to the regional governments in Belgium and controlled a budget of 235
Behavioural Additionality
Jan Larosse
3
16/02/16
million euro in 2003 (40% of the overall Flemish S&T budget). IWT plays an important role
in the new Flemish innovation system. IWT fulfils different tasks; evaluation therefore has
many dimensions and levels.
Although the question of efficiency (use of given resources) is important to IWT
management, is subordinated to the question of effectiveness (attainment of given goals) in
the evaluation of stimulation activities such as R&D subsidies. The first question implies the
improvement of control and capacity management; the second is oriented towards evaluating
desirable outcomes, which is the final policy relevance of the intervention. Do IWT actions
make a difference in the improvement of innovation performance? Evaluation culture in
Flanders is slowly developing, largely as ‘learning by doing’. Evaluation of effectiveness is
not supported with specialised, scientifically based and professionally managed evaluation
departments. To explain this learning process in evaluation, and the integration of the
additionality dimension, we distinguish the different levels of evaluation practice.
Most common in IWT is the project evaluation. IWT is the one-stop-shop for subsidies to
industrial R&D in Flanders. The general feature of the subsidy scheme is its bottom-up
character: it is a permanently open and non-thematic scheme. IWT has a well-developed set
of procedures for project evaluation, based on internal and external referees, to evaluate the
ex-ante effectiveness of the project proposals (ex-post evaluation is starting up). Initially
evaluation criteria were heavily focussed on the scientific qualities and technological risks of
the project, but gradually the economic dimension (‘utilisation’) became equally important,
reflecting the shift from an R&D towards an innovation policy focus. This economic
evaluation doesn’t only concern the financial feasibility of the project or the commercial
prospects for the innovating firm but also the economic return ‘for Flanders’. The most
important economic criterion is that the project must be able to generate an economic value
added that is at least ten times the value of the subsidy.
During the evolution of this IWT evaluation practice also the ‘societal’ qualities of the project
– mainly concerning environmental sustainable development - became part of the evaluation
and selection procedure, although not on the same level as the two main axes. But because of
the formal policy objective to increase the number of projects that support sustainable
development, the latter criteria were formally separated from the standard project evaluation
in a separate arrangement for ‘sustainable technological development’. This arrangement
applies across the existing subsidy schemes and implies an additional evaluation of candidate
projects on criteria of ‘eco-efficiency’. The evaluation gives access to extra support in the
form of a priority ranking in the subsidy scheme concerned and of a financial bonus of 10%
on the project budget.
This evolution illustrates that project evaluation in Flanders is closely linked to general policy
criteria in a bottom-up innovation policy design.
IWT also has an important stake in programme evaluations, because the ‘self-evaluation’ by
the management (up to now mostly ex-post evaluation) is an important input for the policy
(re)design. Programmes (or support schemes) don’t have a thematic focus but are designed on
the basis of characteristics of specific target groups. The ‘SME Innovation Programme’ is
also a bottom-up scheme but supports in addition to research projects also feasibility studies
and allows submitting a broader range of innovation costs to lower the threshold for
innovation in SMEs. Another important programme is the ‘Strategic Basic Research’
Programme that is also non-thematic but organised by a yearly call to form consortia between
universities and companies for longer term projects that are strategically important for
Flanders from an economic and/or social point of view (also non-technological research is
included). Programme evaluation is closely linked to the achievement of these nontechnological objectives.
Innovation policy has developed a complete set of support schemes along the ‘innovation
chain’ that are put available for bottom-up initiative. But it is felt that programme design has
Behavioural Additionality
Jan Larosse
4
16/02/16
to fit the different leverage needs and leverage potentials of various types of innovation
activities and actors. Evaluation criteria try to discriminate access on that general level.
What is still lacking in Flanders is an overall policy evaluation that takes into account the
policy mix as a whole (the complementarity of the different programmes) and assesses the
performance of the innovation system in an international perspective. This is a component of
the organisation of the policy cycle that is not yet formalised, partly because the
institutionalisation of the innovation system is not fully completed yet (a formal Ministry of
Sciences and Innovation will be established in 2004). But the main reason is that the need for
this was not urgently felt because of the strong immersion of policy objectives on the
operational levels by means of differentiated ‘bottom-up’ programmes that give the initiative
to the business and research actors. Formal top-down objectives of government for the
innovation system were inexistent. But the official adoption of the ‘3% target’ will be a
leverage to build more ‘strategic intelligence’ for policy making, starting from monitoring and
evaluation. The coordination of efforts to promote R&D with the federal government, that
still is competent for fiscal instruments to stimulate R&D, is another incentive to assess more
systematically the composition of the policy mix and of the innovation system from a
strategic viewpoint. In particular strategic research has to orient towards the kind of
knowledge economy that has to be constructed for maintaining the welfare position. Guidance
for strategic evaluation on choices to be made is not explicitly formulated yet, but without this
implicit preferences and priorities shape the agenda. Therefore the question of evaluation has
to be linked with explicit analysis of the innovation system and the role of government in
improving performance.
Evaluation practices in Flanders at this moment are rather piecemeal and pragmatic, e.g.
evaluation of the public research institutes or the evaluation of instruments within their own
organisational boundaries. They have a fragmented, organisationally narrow perspective and
lack a firm theoretical basis. The bottom-up character of policies does not necessarily cause
this, because also project evaluation can be put in a broader framework. The stated policy
objectives (promoting innovation for SMEs; promoting cooperation between actors and
technology transfer from research institutes in particular; promoting environmental
technology) are topical and not positioned on a global evaluation scale of the performance of
the innovation system and the strategy of government. Therefore the formal selection criteria
that try to embody these policy objectives in stimulation programmes and projects can only
partly cover the issues of innovation performance on system level. A broader evaluation
approach is needed.
2.2. The new perspective of government on innovation
Because of the limitations of actual evaluation practices it is of great importance to have a
broader view on the behaviour of innovation actors in the perspective of the innovation
system, for the evaluation of the success of policy instruments, be it on project, programme or
overall policy levels.
Firstly, just because there is more than what is captured by the stated policy objectives. More
in particular, the effects of policies have to be considered as the result of an interaction
between public strategies on the one hand and private strategies on the other; companies
responding from their own point of view to the incentives of public instruments. Different
instruments also combine mutually through specific interactions (policy mix). Therefore an
‘ex post’ evaluation of the global impact of government stimulation is needed that goes
beyond a narrow (one organisation) incentive (one instrument) to effect (one objective)
perspective. Policy instruments have to be evaluated as one among many variables that
explain the performance of the targeted actors, and the same goes for the policy mix at the
level of the system performance as a whole. Does policy ‘make a difference’? How to
evaluate results of these interactions?
Behavioural Additionality
Jan Larosse
5
16/02/16
Secondly, the constrained nature of evaluation at project and actor level of a set of private
intentions or results and related public objectives can obscure the broader range of social
returns. These are not fully implied in the actual policy set-up. Innovation seldom is an
individual activity but the result of complex interactive processes. Policy development is just
beginning to discover and explore the huge ‘reserves’ of productivity gains that are linked to
the better operation of the linkages in the innovation system. Innovation policy has to manage
the innovation system from the point of view of the social return; Flemish innovation policy
in particular has to devise a strategy to enhance the performance of the national innovation
system to secure welfare creation for its citizens. This role of government as provider of new
kinds of (soft) infrastructure for innovation actors has to be matched with appropriate
evaluation models.
The ambitions of evaluation policy are therefore closely linked with the appreciation
of the role of government in present innovation systems. This discussion has become
less ‘ideological’, in the sense of dogmatic models that a priori limit this role by virtue
of the merits of mythical markets or take it for granted as the emanation of the general
interest. The present day policy debates have converged towards the agenda of
‘sustainable growth’, the knowledge-based economy being the leverage for more
immaterial types of consumption and for ‘ecological modernisation’ of production.
The Flemish government has launched in 2000 a broad social debate (Colourful
Flanders) and a legislation-crossing mission statement that resulted in 21 programme
points (Pact of Vilvoorde) for the kind of society Flanders wants to build in the 21st
Century. Governments are strategic actors in a largely consensual model of goal
setting, starting with a given set of assets (history matters!) towards an open future
(the reduction of uncertainty).
The implementation of this agenda on EU level (the Lisbon goals) by national governments in
a globalising economy stresses the abilities of societies for structural adaptation and the
strategic role of innovation policies in this respect. In knowledge driven societies that are
dependent on their institutional processing capacity of knowledge and information, the role of
governments changes from a mere provider of framework conditions to an active and
intelligent participant in knowledge creation and distribution; from redistribution of resources
to co-creation of resources being one important actor among others in complex innovation
processes (not denying the central role of business firms in innovation).
How to evaluate the performance of such a government and such innovation policies? There
is a need for ‘transition’ from traditional innovation policies (first and second generation)
towards new kind of innovation policies (third generation) to keep government policies at
pace with the growing demands of the innovation process. At the end a much higher degree of
complexity has to be managed. Evaluating methodologies are an important part of the
capacity to learn and evolve. But it is more difficult to ‘attribute’ effects to specific actions in
a cumulative innovation process of which governments themselves are active participants. At
the moment the proper tools for evaluating third generation policies (indicators, methods,
organisational set-up) are not yet available.
Therefore we need bridging concepts as ‘behavioural additionality’ that have the capacity to
‘internalise’ progressively the new dimensions of interaction between government and other
innovation actors in an evaluation framework. On the one hand this ‘behavioural
additionality’ is a concept that we know and use in traditional evaluation; on the other hand it
exposes new properties that learn us more about the opportunities or limitations of actions by
governments to stimulate innovation performance and transform innovation institutions and
structures.
Behavioural Additionality
Jan Larosse
6
16/02/16
3. About the concept of Behavioural Additionality
For any practitioner of innovation stimulation behavioural additionality has a great intuitive
appeal. The feeling is that allocation of subsidies is more than just a money transfer. There are
different levels of interactions that impact behaviour of the firms. The organisational
interaction between the agency or administration and the receiver encourages the
formalisation of the R&D process in firms. Innovation management is influenced by the
necessities of external evaluation and this procedural approach may also advance IPR
development. The implicit and explicit conditionalities of the granting agency may impact the
course of development of the project as well as the further technology trajectory. Also many
examples can be given of the importance of individual interactions of government advisors
‘doing their job’, with sometimes strategic impacts thanks to their expertise and vigilance.
But often all these impacts are very ‘accidental’, because their potential is not systematically
exploited. They are not visible in evaluation practices, because they have no real legitimacy
and theoretical status. Theoretical arguments, mostly economic theories about the conditions
for welfare enhancement, provide the broad background for policy design.
It is difficult to define a theoretical statute for ‘behavioural additionality’ because of its
emergence at the borderlines of an established theoretical framework that is rather closed. The
original – and still prevailing – concept of ‘additionality’ is linked to ‘welfare economics’ and
the rationale of public intervention in R&D (or knowledge production in general) because of
‘market failures’ (K. Arrow, 1962). Additionality than is the mirror of ‘substitution’ of
private initiative, that means a distortion of free market equilibrium and of competition.
Additionality equals ‘not substitution’, a negatively defined concept. Additionality is
achieved if the coefficient of the subsidy variable in the econometric regression on private
R&D investment is positive; ‘the more the better’.
But we need to define the role of government in a positive way to better manage
innovation systems. This role is more than just compensating under-investment in
R&D without crowding out private financing, and has to be analytically decomposed
into the relevant policy dimensions. We will see that this is done by linking
behavioural additionality to the concept of ‘system failures’.
But here theory has followed practice. The search for the ‘enlargement’ of the
additionality concept has started with the rapprochement to the ‘subsidiarity’ concept,
the division of labour between different governmental levels of intervention. In a
study for the European Parliament (Additionality as a principle of European R&D
Funding, Merit, 1995) Paul David et all introduced a “larger” definition (as opposed
to the “narrow”, financial definition) for evaluation of additionality of a support
instrument extended to ‘complementarity’ with support by other governmental levels.
In Buisseret et all (What difference does it make? Additionality in the public support
of R&D in large firms, IJTM, 1995) the adjective ‘behavioural’ is used for the first
time. It describes new dimensions of evaluation of the impact of government support,
beyond the “direct effect”, notably in the context of impact on firm strategy. The
reference position is “what would have been taken place anyway”, which is a
departure from the optimality theorem of welfare economics. But a thorough
conceptual discussion has been lacking since, because the behavioural aspect was
mainly used to provide an ‘extra’, in surplus of the well-accepted input and output
additionality.
This position cannot be maintained if the role of behavioural additionality in
evaluation of effectiveness becomes more ambitious. Therefore an exploration of the
Behavioural Additionality
Jan Larosse
7
16/02/16
‘limits’ of behavioural additionality as a bridging concept between neo-classical and
system theoretic approaches becomes necessary.
3.1. ‘Additionality’
The intuitive use of ‘additionality’, in a broader sense than intended in comparative static
welfare economics, is grounded in the common guideline ‘making a difference’: the role for
government in improving social welfare. The assumption is that there are ‘hidden’ sources of
productivity that are not tapped by private actors because the necessary institutional
conditions are not gathered, and that these sources or opportunities are growing bigger in the
knowledge economy. Knowledge is proclaimed a special type of economic good that has the
extraordinary characteristic not to ‘wear out’ in consumption. On the contrary, in usage it
grows. But there is a ‘myopic’ theoretical understanding of the functioning of the knowledge
economy, focussed on single actors, that restricts policy learning. The way the ‘collective
productivity’ of knowledge sharing can be organised in a market economy that is based on
individual incentives is not well understood.
One way to analyse this ‘hidden’ resource is with the micro-economic concept of ‘spillovers’
of individual economic activities. In welfare economics these spillovers are just disturbing the
market, causing under-investment in knowledge production. But what if the productivity
gains of these spillovers become a multiple of the productivity gains of the ‘owner’ of the
knowledge (as is the case)? Than subsidies are not just legitimate to encourage the individual
entrepreneur to take more risks, compensating the danger of imitation by free-riders or
compensating bad performing capital markets to take risks. Than new institutional conditions
are needed to encourage ‘co-opetition’ to fully reap the social profits of these spillovers.
Additionality of subsidies becomes more complex because of this shift of focus.
Another way of localising ‘hidden’ sources of knowledge productivity is to start the analysis
the other way around, from the system level, to identify synergies in the system that are not
fully used. Increasing the ‘connectivity’ of the system is another way of saying that
cooperation in innovation has to be stimulated. Other accents are in solving ‘bottlenecks’ or
‘lock-ins’ and balancing ‘mismatches’ between different layers in the innovation system.
Both the micro-economic and the systemic paradigm can be complementary in better
understanding the conditions for unleashing these hidden resources of welfare production and
making government contribute to arrange the institutional preconditions. Additionality
therefore can be redefined in different ways for exploiting resources that are not exploited yet
(additionalities are everywhere in a knowledge economy!), where a ‘positive’ action of
government can make a difference.
But there the problems for evaluation also start. In the conventional understanding of
additionality there was at least a theoretical reference for the desired state of the system: the
‘optimal allocation’ of resources (Pareto-optimality, as a rule achieved by the free market and
the price mechanism). In practice it is unknown how much government subsidies are
sufficient to achieve such a hypothetical allocation in case of market failures. What is possible
is to try to figure out a possible crowding-out or an additionality effect with ex-post
econometric analysis. These econometric models are not comparing actual with Pareto
efficient investment levels, but try to determine if subsidies have partly or completely
substituted private investment efforts in the existing economic circumstances, what is not the
same. ‘Doing better than what would have been without intervention’ only refers to a
previous situation. A lot of problems arise also in modelling the determinants of private R&D
investments, but still the econometric exercise is believed to be one of the only objective tests
of input additionality.
If we turn to a broader understanding of additionality in relation to ‘system failures’, the
range of additionality potential widens but also looses normative ground in economic welfare
Behavioural Additionality
Jan Larosse
8
16/02/16
theory. For one thing, a system of innovation doesn’t only deal with knowledge markets but
also with other institutional mechanisms to coordinate the production and distribution of
knowledge. The science (sub-)system has its own functional logic that is referring to ‘truth’ as
an operator and not ‘value’. And even in the processing of knowledge as an economic good,
the market is one form of coordination alongside others: hierarchy (intra-firm coordination
managed by ‘command’) and networks (inter-firm coordination build on ‘reciprocity’ in stead
of ‘exchange’). The ‘optimal’ system cannot even theoretically be determined because of this
institutional complexity and its non-linear dynamics.
But the evolutionary theory can instruct us about the adaptation behaviour regarding internal
and external pressures. In the knowledge society adaptation means learning. And important
systemic deficiencies are ‘capacity failures’ of actors, which restrain their ability to adapt and
renew. Another problem is ‘path-dependency’ of cumulative learning, because rigidities due
to large past investment in existing knowledge infrastructure can cause ‘network failures’ or
‘lock-ins’ that prohibit innovation. Innovation incentives can be effective (‘additional’) in
changing behaviour or inducing learning. But the norm of behaviour or desirability of certain
outcomes is more a matter of strategic choice than the result of some technical optimalisation.
‘Making the difference’ becomes a kind of quality statement for governments that consider
innovation incentives mainly as instruments for managing the innovation system as a
‘learning economy’. Competence building is a central objective. R&D investment is as much
knowledge ‘creation capacity’ as ‘absorption capacity’, ability to learn about the already
existing or elsewhere newly developed knowledge (which is a multitude of what one creates
at his own). This perspective is still weakly developed in evaluation of the effectiveness of
R&D subsidies. Behavioural additionality then comes into the spotlights.
The concept of additionality has shifted from the narrow context of market failure as a
condition for government to achieve hypothetical Pareto optimality (with the restriction not
crowding out private initiative) to a context of system failure as a condition for government to
improve the system ‘as it is’. This includes the reparation of ‘government failures’
(bureaucratic dysfunctions). Additionality is therefore continuous improvement of the
effectiveness of the policy mix (and not the quest for the optimal state); the excursion of
institutional learning by governments. But in last resort it must make a – measurable!difference on innovation performance.
3.2. ‘Behaviour’
Behavioural additionality is positioned between ‘input additionality’ and ‘output
additionality’ to uncover the ‘black box’ of decision making within firms (the management of
the innovation process) that brings about ‘allocations’ (R&D investments). What is of interest
is not only to know that there is something ‘beyond’ the immediate impacts on input (R&D)
and output (innovation), but to better understand how policy incentives work, such as to
learn to improve them.
The final effect of innovation policy is aimed at innovation output. Input additionality
therefore is only interesting in so far there is a link with output additionality (only
unproblematic in the linear innovation model!) and there is a lack of information to evaluate
innovation output directly. The time lag between subsidy and innovation outcome is therefore
a strong argument for input additionality evaluation. In this respect (improved) R&D
investment or R&D cooperation are used as ‘proxies’ for output additionality. But this is not
evident if output is not linearly linked to the preceding stages: output is not fully determined
by the previous decisions. The number of causal factors increases coming near to the market;
it is increasingly difficult to attribute the proper impact to the R&D-investment (let alone the
R&D-subsidy). Many output solutions are possible because of the complexity of the process
and because the innovation actors learn during the process. Innovation output can therefore be
something totally different - by surprise – because of this unpredictable character. Therefore
the behavioural additionalities, increasing the competences of the actors and the
Behavioural Additionality
Jan Larosse
9
16/02/16
conduciveness of the system, are the more important additionalities, because they create a
generic capacity that has a lot of spillovers for further innovation. Behavioural impacts are
enduring effects because they are internalised and reproduced in the behaviour of agents (not
one-time effects) and have structural or institutional impacts on the system.
The nature of the innovation behaviour is broader than market behaviour, because innovation
systems are broader than markets. It concerns all kinds of social behaviour that affect the
working of the innovation system. The behaviour is not limited to natural persons but
describes rules of conduct or routines that also are embodied in institutions, e.g. firms (firm
behaviour). Different institutions embody different forms of ‘routines’ that are sensitive to
other kinds of incentives. Institutional theory describes how markets react to price signals,
hierarchies to rules and commands, networks to shared convictions. Innovation policies will
have to understand and use the strengths and weaknesses of these different institutional
coordinating mechanisms to improve the ‘behaviour’ of the innovation system as a whole.
Therefore ‘behaviour additionality’ can claim to be the more general concept that
encompasses the other kinds of additionality that all have to do with specific kinds of
behaviour, e.g. input additionality covering ‘investment behaviour’. Investment behaviour is
also much broader than just the resource allocation in static markets. In dynamic perspective
strategy development handling the uncertainties about future markets is very important:
‘portfolio composition behaviour’ is therefore an additional dimension in R&D investment
that deserves all attention and is only revealed as a subject for improvement of policy making
in so far it is made visible by new conceptual frameworks. Portfolio management theory and
strategic management theory have to be ‘translated’ in the multi-institutional context of
innovation systems.
3.3. Kinds of additionality
Depending on the policy perspective different types of additionality can be distinguished in
the literature. The question whether behavioural change is a dimension of these types of
additionality or the reverse, that behavioural additionality is structured into these dimensions,
is a matter of perspective.
- From the point of view of the phases in the innovation process (the ‘arrow of time’), we can
make the sequence of input, throughput (or internal process), and output. Sometimes
behavioural additionality is narrowed to (internal) process additionality.
- From the point of view of the organisational level of the impact, we can see different
effects on project level, firm level and system level.
- From the point of view of the types of productivity gains (or types of economies) we can
distinguish additionalities of scale, scope and speed.
- From the point of view of the interaction level of agencies with the innovation actors we
can have operational and strategic additionalities, direct and indirect additionalities (spillover
additionalities)
It might be confusing to substantiate each specific behavioural additionality effect as a
separate type (e.g. ‘cooperation additionality’, ‘portfolio composition additionality’), but in
this stage of research the advantage of making them visible as subjects for policy action is
more important.
The more important ‘policy additionalities’, which can be achieved if government can have a
socially beneficially impact on private strategies, are also a potential strategic area for making
a difference, if the necessary conditions are present (horizontal innovation policy).
Behavioural Additionality
Jan Larosse
10
16/02/16
Behavioural Additionality in many dimensions
Organisational
Project
level
Company /
Strategy
Process level
Input
R&D-budget
Scale
Time
Risc
System
Knowledge spillovers
Portfolio composition
Throughput
- Internal
- External
Project management Innovation
management
IPR behaviour
Synergy in proj.
family
Cooperation
Alliances
Networking / clustering
Strategic autonomy
Relations with
VC/financiers
Quality label
Output
Procesinnovation
Productinnovation
Training
Localisation
Strengthening core
or
differentiation
Improved environmental
impact
Specialised knowledge Human capital
/ know-how
4. About the measurement of behavioural additionality
The broadening of the additionality concept makes it more complex and multiplies the
evaluation and measurement problems. The most known problems that have been identified
are the problem of ‘counterfactual’ evaluation and the problem of ‘attribution’ of cause and
effect. The first problem is inevitable because time is irreversible and conditions are never
completely comparable between a situation with and without a subsidy. Control groups of
firms without subsidies to measure the impact by comparison are very difficult to compose;
the reasons for applying (or for denying) subsidies are linked with other characteristics that
already differentiate these firms. The second problem is more general the problem of
separating one impact – a subsidy - in a multi-causal phenomenon.
These problems are not to be solved with some high-flying measurement technique, but the
strategy can be to use complexity itself as a solution method, using the complementarity
between different forms of partial solution. Measurement is objectivation without
mystification!
The evaluation of the effectiveness of innovation policies poses two kinds of specific
evaluation challenges: the assessment of the behavioural additionality of instruments on the
interactions in the system, and the assessment of the behavioural additionality on social
return. This is because we consider the mutual reference of knowledge production (creation)
Behavioural Additionality
Jan Larosse
11
16/02/16
and knowledge consumption (diffusion and use) as being the main engine of cumulative
growth of the knowledge economy
The first item – interaction additionality - concerns the opening of the ‘black box’ of the
interaction between agency and firms, in particular the effects of instruments on the
decision processes and private strategies about R&D (one-time and enduring effects). The
different types of behavioural effects of the instrument need to be identified and assessed for
their learning leverage. This will preferably be in quantitative form in order to be related to
innovation performance.
But this evaluation finds also an end in itself, supporting policy design to improve the
additionality enhancing qualities of the questioned instrument and the policy mix. How this
can be done is by institutional learning in the form of real life ‘experiments’ and studies on
the basis of hypothesis testing (thought experiments in policy debate or econometric tests).
The use of the control group and of appropriate ways of ‘revealing’ the ‘true’ behaviour are
important items for this assessment.
The second item – spillover additionality - concerns the estimation of the contribution of
policy (through various kinds of behavioural additionality) to the social return (that is the
global outcome of the innovation process). Additionality (documented as behavioural changes
induced by policy) appears as an extra social return. This means we first have to find better
ways to know social return, the combined direct and indirect returns of investments (e.g. in
cluster performance evaluation). The problems of disentangling total factor productivity
illustrate the challenge of measuring the different components of growth. The challenge is to
identify the effect of the behavioural changes on social returns (TFP). Starting with
inventorying these changes and with tracing the spillover channels.
A related problem in the evaluation of behavioural additionality in terms of social return is
that because of the dual nature of knowledge as consumption and production good one has to
beware for ‘double counting’. E.g. the ‘soft’ additionalities of learning are often the other side
of the ‘hard’ additionalities of financial support if they simply come along without any
specific additional intervention. If they would have occurred anyway they are the soft side of
the same coin, while additionality targets additional coins! Another related problem is that
negative additionalities of support (administrative burden and government failures) have to be
subtracted.
Again the comparative assessment with a control group or counterfactual scenario seems of
great importance to understand magnitudes of additionality.
In this assessment we have to be able to compare different instrument on their ‘relative’ and
‘absolute’ additionality. If additionality is relatively bigger in terms of behavioural effect, it is
the absolute additionality in terms of innovation output that is the reference! A big change in
behaviour in a small actor may result in less absolute additionality compared to a situation
were the social return of smaller changes (e.g. in a cluster context) may have a broader basis.
5. Policy learning
The potential impacts on the innovation system of new evaluation methods that focus on
behavioural additionality are important, sometimes more important than the behavioural
impacts of the existing policy instruments that they can reveal. The purpose of evaluation is
indeed to change the policy design.
In the case of behavioural additionality it is more and more policy practice that defies
evaluation methodology. In different ways stimulation policies are internalising behavioural
additionality strongly in the policy instruments. E.g. IWT has introduced the instrument of
‘feasibility study’ as a de facto obligatory precondition to apply for an ‘innovation project’;
this way encouraging the development of innovation management skills and complementary
knowledge building, in the firm and in cooperation with knowledge providers. Another strong
example in Finland is the set-up of an integrated support infrastructure (INTRO) to stimulate
Behavioural Additionality
Jan Larosse
12
16/02/16
the pre-seed market by Sitra, the Finish public venture-capital fund. In cooperation with
Tekes and a network of specialised actors it provides the complementary inputs
(technological opportunity, finance, business experience, managerial skills) to launch new
start-ups. Financing of business-plans or intermediation to find experienced people with
commercial background: the behavioural additionality to the new entrepreneurs - but also to
all complementary actors - is enormous.
Also in less obvious cases there is a potential for improving the behavioural additionality in
innovation processes resulting from government intervention, on the condition that policy
makers consider each activity in a system context. In the short term this can result in
incremental improvements in the existing programmes (what can be a leverage in awareness
creation, selection procedures, contract negotiations to follow-up of mile-stones and
commercialisation?), but in the longer run the consideration for the interaction effects of
government and firm strategies are the basis of improving the system coherence and
overviewing the dynamic properties of the system as a whole. Policy learning therefore has to
take account of behavioural additionality.
In Belgium the policy mix of R&D stimulation by fiscal and subsidy instruments is
constrained by an institutional divide between federal and regional governments. A better
understanding of the behavioural additionality properties of each instrument can help to find a
better ‘division of labour’ and a complementary fine-tuning of the instruments, according
differences in their behavioural additionality towards different types of firms.
The introduction of new evaluation methodologies that consider behavioural additionality as a
touching stone of effectiveness in a system perspective is a factor of knowledge
intensification of the management of the innovation system itself and of a transition towards a
third generation innovation policy.
6. Organisation of the work programme in Flanders
IWT-Flanders is an operational organisation with limited resources for methodological work.
The strategy is to cooperate with national and international partners to advance in the
elaboration of evaluation methodology and of empirical evaluations of behavioural
additionality. In the past the subject has been on the agenda in explicit initiatives such as
international conferences. As early as 1996 a Conference was organised in the framework of
the Six Countries Programme on the subject ‘R&D Subsidies at Stake: In Search of a
Rationale for Public Funding of R&D’, where the discussion on ‘behavioural additionality’ as
a rationale was introduced. In 2003 a follow-up conference on ‘Innovation Policy and
Sustainable Development: Can Innovation Incentives make a Difference?’ advanced the new
approach of behavioural additionality to incentives for innovation that enhances sustainable
development. IWT practice also has been the breeding ground for the development of new
policy (the arrangement for sustainable technology development) that goes in this direction.
This is the reason that IWT can play a role as forerunner in different international initiatives
(OECD, TAFTIE).
Since 2002 the evaluation of additionality has been put on the agenda of the IWT Observatory
by the IWT management. The working programme that is now being constructed tries to
exploit the complementarities of different methodologies and partners in view of a
comprehensive methodology for policy learning.
- Exploratory and case study research.
In 2003 an exploratory study was conducted to test a new survey instrument for collecting
information of different types of additionality and their effects on performance. This was
combined with a case study type of investigation of the differences in additionality potential
of different classes of firms. Four categories of firms were targeted according the dimensions
of scale and research-intensity (see B. Clarysse et al: Measuring Additionality of R&D
Subsidies by Surveys: Towards an Evaluation Methodology for IWT-Studies).
Behavioural Additionality
Jan Larosse
13
16/02/16
A next step will be to extend this pilot to a large-scale survey and to construct a database of
additionality characteristics for statistical analysis. Special attention will be devoted to the
construction of a control group database. Structured interviews with firm management are a
complementary instrument to deepen the understanding of internal and interaction processes
that are important to assess additionality potential.
- Econometric work
In 2004 the ‘Support Point R&D Statistics’ at the Leuven University to examine the input
additionality of R&D-SUBSIDIES will conduct a ‘classic’ econometric research project. The
database of R&D and subsidy data will be enriched by all relevant data from the IWT
database (e.g. on cooperations) to measure behavioural differences. The specification of the
econometric model and the specification of the survey questionnaire will be coordinated to
mutually benefit (improving the explanatory value with new behavioural variables).
In this work international exchange of experience with other research groups or agencies is
very valuable. To extend the empirical base for common work a similar data collection is very
useful.
- International comparative work
Besides exchange of best practice in methodologies and the construction of a common
empirical reference base, international comparative work is very important to assess the
impact of different institutional set-ups in the organisation of innovation stimulation.
Understanding the qualitative particularities that can explain certain strengths and
weaknesses, in connection with quantitative results, is a very important way of policy
learning.
In this process an international community of practice is formed that can be an asset in actual
national evaluation exercises. A natural extension of this cooperation could be the set-up of an
international peer-review system of evaluators that is another complementary element in the
evaluation set-up.
- Internalisation in existing evaluation procedures
The organisation of an ex-post evaluation is a cumbersome and costly exercise. It is important
to make sure that the results are useful and used. The mandate for this kind of exercise
coming from the policy makers is vital. The preparation and implementation of the evaluation
has to involve the management and the advisors of IWT. They are also an important source of
information.
But the internalisation of this evaluation will have two directions. First, enhance the
instruments and the organisation of the interaction process with the ‘clients’ in a more
system-aware sense. Second, include new indicators for the evaluation of behavioural
additionality as part of the existing evaluation procedures. This material will feed-in in the
evaluation methods above described and increase the learning capacity of the IWT.
Behavioural Additionality
Jan Larosse
14
16/02/16