Active labour market measures
and entrepreneurship in Poland
Rafał Trzciński
Impact Evaluation Spring School
Hungary, 18.05.2011
case 1 – Evaluation of @lternative II project
• Objective of the project: preventing unemployment among young people.
• The territorial scope: 59 poviats of Poland (NUTS 4) with unemployment
rate above 20% and high unemployment among young people.
• Eligibility:
• young unemployed (27 or younger),
• registered at the labour office.
• Total number of beneficiaries: 5 657.
• Budget: € 4 090 702.
• Type of services: pre-training assistance (recruitment, needs assessment,
guidance); vocational training services linked with ECDL, both at basic and
advanced level, as well as related areas; practical training (temporary
employment/ on the job training organised under the agreements signed with
employers; training allowances; vocational guidance and post training
assistance; job-broking.
• Period of implementation: 2005-2006.
Evaluation problem
Efect: lower
unemployment
@lternative II
project
Selection
Factor
to the x1
project
SELECTION
BIAS
?
Problem: High
unemployment
rate among
young people
Self-selection
Factor xn
to the project
The selection bias problem in
control/comparison group approach
To estimate the impact of the intervention
we cannot simply compare beneficiaries
(treated) with those who did not
participate in the project (non-treated)
 This is because of factors affecting both
participation and outcomes.
 If we don't control for those factors, we
can overestimate or underestimate the
impacts(picking the winners/ picking the
losers).

Data used in the evaluation
PULS System, which:
• is used for services for the unemployed
• is present in approximately 90% of Poviat Labour
Offices in Poland (2006)
• provides a wide range of data on each
unemployed person (socio-demographics,
employment characteristics, previous qualification
improvement, skills etc. ),
• includes a detailed history of unemployment and
other activities on each person (registration in
the office, deregistration, trainings, use of the
benefits, etc.).
Data collection
• We collected data from 55 of the 59 Poviat Labour
Offices involved in the project.
• In total we managed to identify 5 065 participants
of the @lternative II project (90% of all
beneficiaries).
• Moreover we collected data on 126 633 persons
(non-treated), which meet the formal conditions for
eligibility for the project (registration in the labour
office, age condition).
Variables
Socio-demographic characteristics
 Sex
 Age
 Marital status
 Single parenting
 Number of children
 Education
 Poviat
Previous qualification improvement
 Number of training courses, in which
the person participated during the
one year prior to participation in the
project
 Total number of days spent on training
 Having a work placement before
participating in the project
Skills
 Possession of driving license
(B category)
Employment characteristics
 Profession (ten categories)
 Number of days of work
 Number of professions (in total)
 Number of days being unemployed
before participating in the project
 Number of days receiving the
unemployment benefit before…
 Number of job offers during the one
year period before…
 Number of days participating in
subsidised work
 Number of days of permanent
unemployment (during the two years
period before…)
Motivation to find a job
 Percentage of showing up in the
Poviat Labour Office,
 Having the right to unemployment
benefit
Back to the selection problem…
0,259
0,132
0,024
0,012
0,075
0,694
0,072
0,121
0,032
0,087
21,932
159,267
0,251
Eligible non
participants
0,433
0,242
0,189
0,033
0,260
0,382
0,044
0,086
0,077
0,272
23,302
366,093
0,145
Standardized
difference in %
-37,1
-28,6
-55,5
-13,9
-50,9
66,0
11,6
11,6
-20,1
-37,6
-66,1
-42,9
26,8
industrial wokers/craftsman workers
0,065
0,192
-38,7
without occupation
v45 Number of days being unemployed before participating in the A.
v46 Number of days receiving the unemployment benefit before…
without the right
0,248
473,867
299,207
0,380
0,189
819,623
447,247
0,262
14,4
-59,1
-42,3
25,4
< 1 month
0,062
0,072
-4,3
>1 month but <3 months
0,121
0,153
-9,3
0,437
0,512
-15,0
233,624
330,349
-39,6
Variable
Beneficiaries
v47
Having the
right to
unemployment
benefit
v17
Profession
Education
v3 Sex (male)
v5 Marital status (married)
Primary
Lower secondary
Basic vocational
v6
Secondary
Post-secondary
Higher
v7 Single parenting
v8 Number of children
v9 Age
v10 Number of days of work
technicians
>3 months
Number of days of permanent unemployment
v49
(during the two years period before…)
Bearing in mind the assumptions…
Conditional Independence Assumption
Population A
Counterfactual
Treatment
action
Population B
Counterfactual
Treatment
action
• We assume that if we can control for observable differences in characteristics
between the beneficiares and non-treated population, the outcome (observable
change) that would result in the absence of treatment (counterfactual action) is
the same in both populations.
• Ergo, we assume that unobservables do not affect the outcomes!
Propensity score matching (1-1; nearest neighbour)
Beneficiaries
(N=5 065)
Eligible
non participants
Control
group
(N=126
633)
(N=5 065)
ps= 0,8
ps= 0,1
ps= 0,4
ps= 0,5
ps= 0,9
ps= 0,6
ps= 0,2
ps= 0,1
ps= 0,3
ps= 0,2
ps= 0,01
ps= 0,3
ps= 0,8
ps= 0,9
ps= 0,4
What we have achieved using PSM?
0,259
0,132
0,024
0,012
0,075
0,694
0,072
0,121
0,032
0,087
21,932
159,267
0,251
0,065
0,248
Eligible non
participants
0,433
0,242
0,189
0,033
0,260
0,382
0,044
0,086
0,077
0,272
23,302
366,093
0,145
0,192
0,189
Control
group
0,258
0,138
0,025
0,010
0,073
0,693
0,076
0,122
0,031
0,089
21,952
150,804
0,258
0,061
0,242
Standardized
difference in %
0,3
-1,6
-0,1
1,3
0,7
0,3
-2,0
-0,2
0,4
-0,2
-1,0
1,8
-2,0
1,3
1,4
473,867
819,623
483,460
-1,6
299,207
447,247
304,967
-1,6
0,380
0,262
0,378
0,5
< 1 month
0,062
0,072
0,062
0,0
>1 month but <3 months
0,121
0,153
0,123
-0,5
0,437
0,512
0,438
-0,1
233,624
330,349
267,151
-13,7
Variable
Beneficiaries
v47
Having the
right to
unemployment
benefit
Profession
Education
v3 Sex (male)
v5 Marital status (married)
Primary
Lower secondary
Basic vocational
v6
Secondary
Post-secondary
Higher
v7 Single parenting
v8 Number of children
v9 Age
v10 Number of days of work
technicians
v17
industrial wokers/craftsman workers
without occupation
Number of days being unemployed before
v45
participating in the project
Number of days receiving the unemployment
v46
benefit before…
without the right
>3 months
Number of days of permanent unemployment
v49
(during the two years period before…)
Employment rate
Impact
50%
45%
40%
35%
30%
25%
20%
15%
10%
5%
0%
0
1
Beneficiaries
2
3
4
Control group
5
6
7
8
9
10
Non-treated population
11
12
Month
Source: Ex-post evaluation of Phare 2003 Economic and Social Cohesion – Human Resources Development component,
PAED, Warsaw 2007
Impact
Month after
participation in the
project
A.
Beneficiaries
B.
Control group
C.
Eligible non
participants
A-B:
Impact
T test
A-B*
0
0,00%
0,00%
0,00%
0,00%
-
1
21,86%
20,04%
18,26%
1,82%
+
2
25,19%
22,29%
20,83%
2,90%
+
3
27,84%
24,80%
22,98%
3,04%
+
4
30,48%
26,87%
24,94%
3,61%
+
5
32,73%
29,10%
26,56%
3,63%
+
6
35,28%
30,86%
28,09%
4,42%
+
7
37,67%
32,60%
29,59%
5,07%
+
8
39,80%
34,43%
30,84%
5,37%
+
9
41,64%
35,60%
32,00%
6,04%
+
10
43,65%
36,58%
32,97%
7,07%
+
11
45,17%
38,28%
34,04%
6,89%
+
12
46,22%
39,23%
34,93%
6,99%
+
Cost-benefit analysis
Money spent
€ 4 090 702
Employment rate after 12 months (observed change)
46%
Number of beneficiares that found the job
2 602
Average cost
€ 1 572
Impact
7%
Estimated number of beneficiares that found the job
due to the participation in the project
396
Average cost
€ 10 330
(Counter)example 2 –
Entrepreneurship promotion project
• Objective of the project: encouraging business
activities among unemployed people.
• Beneficiaries: unemployed people (with priority
to young job-seekers).
• Type of services: initial business training;
guidance on conducting economic activities;
training allowance; relevant specialised training;
coaching after setting up a business.
• Time of implementation: 2004-2005.
• Evaluation framework: the same approach as
in the @lternativa II exaple (the same
methodology, source of data, analysis...).
Self-employment rate
Impact?
30%
25%
20%
15%
10%
5%
0%
0
1
2
3
Beneficiaries
4
5
6
7
Control group
8
9
10
11
12
Month
Source: Ex-post evaluation of Phare 2002 Economic and Social Cohesion – Human Resources Development component,
PAED, Warsaw 2006
Lessons learned/points for the discussion

What data we were lacking in both examples?
◦ Missing covariates? (Are our assumptions plausible?)
◦ Missing outcome variables?
What do we know and what we don't know after
completing the evaluation (towards theory based impact
evaluation)?
 How we could modify the plan of the evaluaton to get
more insight on impacts (targeting issue)?
 What is the avaibility of systems such as PULS in other
EU countries (looking for possibilities of implementing
IE)?
 What is the utility of data collected in public statistics?
Do we need new data systems for IE or maybe we need
to modify existing ones? (towards more systematic
discussion on IE planning).

Thank you!!!