Ireland
Evaluating the impact of
enterprise/industrial policy
supports: developing new methods
and approaches
Dr Helena Lenihan
Department of Economics
Kemmy Business School
University of Limerick
Helena.lenihan@ul.ie
Our world has changed!
Industrial Policy in Europe
• A stronger European Industry for Growth
and Economic Recovery (10th Oct 2012)
• The pillars for a reinforced industrial policy
– Investment in innovation
– Better market conditions
– Access to capital
– Human capital and skills
Progress report from
the ‘Action Plan’ (Q1-2012):
• Launch of a new Potential Exporters
Division in Enterprise Ireland
• €150 million Partial Credit Guarantee
Scheme.
Progress report from
the ‘Action Plan’ (Q1 and Q3-2012):
• A second call under Innovation Fund
Ireland
• Introduction of funding programmes
targeting female entrepreneurs and also
for business clustering
• Pilot Industry-led clustering programme
Progress Reports on the
Action Plan for Jobs
Departments and agencies have delivered to
completion:
Q2-2012 -A completion rate of 94%.
Q3-2012-A completion rate of 87%
Old wine in new bottles?
In 2011, 82 Enterprise Ireland-backed
entrepreneurship programmes took place
Specifying Policy objectives
Lack of Enterprise policy
evaluation nationally
Compared to the UK, Ireland has been very
poor in building datasets to inform policy
design and evaluation (Ruane, 2012)
Defining Policy Evaluation
• Exploring the counterfactual
• What works best under what circumstances?
• Comprises analytically rigorous attempts to
determine the impact of public policy
initiatives.
• Key challenge to isolate the impact of a
particular policy intervention.
Why Evaluate?
• Impact
• Value for money
• Leads to policy improvements
Lots of calls recently for
Policy Evaluation (1)
2012 Action Plan for Jobs – “We will
complete evaluations of all agency
programmes and schemes to ensure that
they are adequately addressing the needs of
today’s manufacturing businesses”
(Action 7.1.4).
Policy Evaluation
Let’s keep it on agenda
- Let’s do it properly this time
– not just another fad
(1) Current evaluation frameworks/metrics
are generally lacking vis-à-vis the
evaluation of ‘newer’ enterprise policies
coupled with a move from ‘old’ towards
‘new’ business models.
(2) Illustrate how ‘LOGIC’ models can be
extended to account for broader ‘societal’
impacts.
The ‘world’ for the firm
has also changed (1)
•
Old business model characterised by:
– desire for business stability
– focus on core business activity
– efficiency attainment via standard
procedures
– focus on developing in-house
specific competencies
The ‘world’ for the firm
has also changed (2)
• ‘Evolutionary’ approach
– Multi-level perspective
– No firm is an island
• Enabling conditions for enterprise start-up
and growth.
• R&D; innovation, commercialisation of
capital, networking and clustering,
entrepreneurship and education at the heart
of enterprise policy.
“New” enterprise/business
model focuses on:
•
•
•
•
•
•
Innovation and novelty and flexibility
Lack of predictability
Operates as part of a network
Combines internal & external ‘know how’
Operates as part of supply chain models
Invests in Human as opposed to physical
capital
• Knowledge based economy and rests on
intangible assets
What does the new approach
mean for policy evaluation? (1)
Evaluation metrics should include as many
as possible of the following:
•
•
•
•
Socio-economic fabric of locality or region
Extent of additionality
Opportunity cost issues
Impact on ‘wider publics’ (Public Interest
Evaluation’) and a concern with social
impact and value
What does the new approach
mean for policy evaluation? (2)
• Impact on sustainable development
• Impact on average local/regional wage
rates within economy.
• Displacement and crowding out effects
across firms.
• Capacity building
• Linkages between firms
• Multiplier impacts
What does the new approach
mean for policy evaluation? (3)
• Knowledge creation through institutional
diversity and pluralism.
• Impact of the environment shaping the
economy upon the dynamics of
entrepreneurship/enterprise activity.
Logic model-potential to make a
worthwhile methodological contribution?
• Provide a guide as to what should be
assessed and measured.
• Incorporate multi-level (layered) effects.
• Compare ideal against actual operation in
a programme.
• Not only look at impact at level of firm but
has potential to consider wider economy
and even society effects.
Technology
Activities
Experience
Personnel
Time
Inputs
Events &
Work-shops
Meeting
Email
Partners
Box 1
Money
Short-Term &
Medium-Term Outcomes
Figure 1: Logic model
(chain of cause-and-effect)
Co-op
Interfirm
Academic
Agencies
Change in Mindset: Box 2
Increased
Increased
Collaboration:
knowledge;
Learning:
Innovation;
Skills
Firm Profitability
Firm Survival.
Box 3
Increased
Attractiveness
of the Local
Economy;
Risk with
Jobs
sharing
info
Unproductive
relationships;
Adapted from Lenihan (2011)
Long-Term Outcomes
Box 4
Increased
Network &
Economic activity;
Improved
Knowledge Transfer
Local, Regional
National
Box 5
Unproductive
Networks:
Displacement
Figure 1: Logic model
(chain of cause-and-effect)
Public Policy
interventions
Box 6
Positive on
society Improved
infrastructure;
Negative on
society increased
traffic congestion;
consumer prices
Box 7
Entrepreneurial
activity in Society
Box 8
Economic
Development
Innovation;
Welfare Effects
Adapted from Lenihan (2011)
Source: Lenihan, H. (2011) ‘Enterprise policy evaluation: Is there a ‘new’ way of doing it?’
Evaluation and Program Planning, 34 (4), 323-332.
Conclusion and way forward
for policy evaluation (1)
• Challenge: move away from evaluation
methodologies that are purely concerned with
narrowly defined economic impacts and
measure impact purely at level of assisted
firms.
• Bottom line: firm and economic activity more
broadly do not operate in a silo but operate as
part of a ‘holistic’ system.
Conclusion and way forward
for policy evaluation (2)
• Greater investment should be made with
regard to ex-ante evaluation.
• Evaluation as a ‘science’ is lacking when it
comes to theory.
• Caution against the use of datasets
originally collected for different purposes.
Conclusion and way forward
for policy evaluation (3)
• Old Chestnut: ‘How’ to evaluate (single
biggest challenge)
• Logic model-potential
• March 2012-Irish Government Economic
and Evaluation Service (IGEES)
Conclusion and way forward
for policy evaluation (4)
• Make the ‘Evaluations’ publicly available
• Most important specify policy objectives
The way we evaluate enterprise
policy impact must also change!
Thank you
Comments/Questions?
Helena.Lenihan@ul.ie