Framework Contract
This project is co-financed by
the European Union and the Republic of Turkey
Specific Contract N° TR07H4.02-01
Providing Technical Assistance for
the First Interim Evaluation of Human
Resources Development Operational
Programme
INTERIM EVALUATION REPORT
November 2011
1
“The content of this publication is the sole responsibility of the Consultant and can in
no way be taken to reflect the views of the European Union.”
2
Contents
Abbreviations ........................................................................................................................................... 7
1. Executive Summary............................................................................................................................. 9
2. Introduction ........................................................................................................................................ 12
2.1 Overview of the HRD OP priority structure, measures and objectives ....................................... 12
2.2 Purpose of the Interim Evaluation and evaluation questions ...................................................... 13
2.3 Outline of the Interim Evaluation Report ..................................................................................... 15
3. Main methods .................................................................................................................................... 16
3.1 General approach to the Interim Evaluation ............................................................................... 16
3.2 The evaluation methodology subject to the evaluation questions .............................................. 17
The Socio-Economic Study in Annex A1 ...................................................................................... 17
Mapping of the institutional environment and studying current practices ..................................... 17
The logic of the Report and the way the evaluation question was addressed ............................. 18
3.3 Challenges and solutions found .................................................................................................. 25
4. Main findings, conclusions and recommendations ............................................................................ 26
4.1 Examination of the Management and control structures of the HRD OP and SWOT ................ 26
4.1.1 Operating Structure (OS) .................................................................................................... 27
4.1.2 The operation beneficiaries ................................................................................................. 30
4.1.3 The Social Partners and the Chambers .............................................................................. 34
4.2 The Monitoring Arrangements and the Monitoring Information System with respect to the Grant
Monitoring.......................................................................................................................................... 34
4.2.1 Central Grant Monitoring Teams ......................................................................................... 34
4.2.2 Regional Grant Monitoring and Technical Assistance Teams (RGMTTs) .......................... 36
4.2.3 The Monitoring Information System ..................................................................................... 36
4.3 The use and financial allocation of financial assistance and the financial management of the
budget ............................................................................................................................................... 37
Overall comments on financial management ............................................................................... 37
Commitment and disbursement data ............................................................................................ 37
National co-financing and EU contribution ................................................................................... 38
Payments to contractors ............................................................................................................... 38
Shorter implementation periods and decommitment risk ............................................................. 38
4.4 Review programme outputs and results in relation with the expenditures incurred at priority and
measure levels; in the light of financial and physical progress ......................................................... 40
4.5 Examine the efficiency of Operation Selection Criteria ............................................................... 41
4.6 Assessment of local absorption and implementation capacities................................................. 42
Assessment of local absorption capacities ................................................................................... 42
Assessment of local implementation capacities ........................................................................... 44
4.7 Analysis of the accuracy of the budget allocations under measures .......................................... 44
4.8 The study on Indicators ............................................................................................................... 45
Conclusions and recommendations on the study of Indicators .................................................... 46
4.9 Overall assessment of the HRD OP ........................................................................................... 48
4.10 Recommendations .................................................................................................................... 49
3
5. Annexes: Overview ............................................................................................................................ 51
Annex A1. Socio-Economic Study ......................................................................................................... 52
1 Introduction..................................................................................................................................... 52
2 Demographic Indicators ................................................................................................................. 53
3 Main Economic Indicators .............................................................................................................. 58
4 Labour Market Indicators ............................................................................................................... 64
5. Indicators on Education ................................................................................................................. 68
Appendix 1. Background Tables ....................................................................................................... 72
Annex 2. Major developments since 2007 ............................................................................................ 92
1. Major Developments since 2007 at National Level ....................................................................... 92
Implementation of Partnership Approach ..................................................................................... 92
Sectoral Monitoring Committee .................................................................................................... 92
Services Provided by Different Institutions Related to HRD-OP Target Groups .......................... 92
Collaborations amongst Related Institutions ................................................................................ 93
Evaluation of Institutional Collaborations ...................................................................................... 94
Development in Higher Education ................................................................................................ 94
Removing Coefficient System to Promote of Vocational and Technical Secondary Schools’
Students to Universities ................................................................................................................ 94
Vocational Qualification Authority (VQA) ...................................................................................... 94
Major HRD-Related Projects Implemented since 2007 ................................................................ 95
Major Outcomes and Evaluation of the HRD-OP Projects ........................................................... 96
Other HRD Related Projects ......................................................................................................... 97
Yearly Major Developments in the Implementations of the HRD-OP ........................................... 98
Major Developments in the Implementation of the HRD-OP in 2007 ........................................... 98
Major Development in the Implementation of the HRD-OP in 2008 ........................................... 100
Major Developments in the Implementation of the HRD-OP in 2009 ......................................... 101
Major Development in the Implementation of the HRD-OP in 2010 ........................................... 102
2. Major developments since 2007 at EU level ............................................................................... 103
2.1 The Lisbon Strategy .............................................................................................................. 103
2.2 The Europe 2020 Strategy .................................................................................................... 104
2.3 The EU Employment Guidelines and the Broad Economic Policy Guidelines ..................... 105
3 Coherence in objectives ............................................................................................................... 106
3.1 The Strategic Coherence Framework ................................................................................... 106
3.2 Coherence between HRD OP and the main EU strategy ..................................................... 107
Annex 3. Review of the results of ex-ante evaluation ......................................................................... 111
1. Partnership Approach ................................................................................................................. 111
2. The study of the Current Situation .............................................................................................. 111
3. The Effectiveness of the planned strategy .................................................................................. 113
4. Indicators ..................................................................................................................................... 113
5. Sustainability of Results .............................................................................................................. 113
6. Absorption Capacities ................................................................................................................. 113
7. The ex-ante evaluation’s conclusions and recommendations regarding HRDOP implementation
........................................................................................................................................................ 114
4
Annex 4. Study of the HRD OPs Effectiveness and Efficiency ........................................................... 117
1 The use and financial allocation of financial assistance and the financial management of the
budget ............................................................................................................................................. 117
Background information .............................................................................................................. 117
2. Review programme outputs and results in relation with the expenditures incurred at priority and
measure levels; in the light of financial and physical progress ....................................................... 119
Statistical data............................................................................................................................. 119
Analysis ....................................................................................................................................... 120
3. Examine the efficiency of Operation Selection Criteria ............................................................... 121
Background information. ............................................................................................................. 121
4. Assessment of local absorption and implementation capacities ................................................. 123
Statistical data on local absorption capacities ............................................................................ 123
5. Analysis of the accuracy of the budget allocations under measures .......................................... 126
Annex A5 The Study of Indicators ....................................................................................................... 128
1. Overview and Introduction: The study on Indicators and the Relevance of the HRD OP .......... 128
PART I: Methodology and approach to the study on indicators ...................................................... 128
I.1. Methodological note on Evaluation of reform programmes and strategies in Human
Resources ................................................................................................................................... 128
I.2 Approach and limitations to the study of indicators ............................................................... 138
PART II: Output Indicators .............................................................................................................. 143
II.1 Priority axis 1 ........................................................................................................................ 143
II.2 Priority axis 2 ........................................................................................................................ 161
II.3 Priority axis 3 ........................................................................................................................ 167
II.4 Priority axis 4 ........................................................................................................................ 174
II.5 Priority axis 5: Technical Assistance .................................................................................... 177
PART III: Result Indicators .............................................................................................................. 180
III.1 Priority axis 1 ....................................................................................................................... 181
III.2 Priority axis 2 ....................................................................................................................... 197
III.3 Priority axis 3 ....................................................................................................................... 204
III.4 Priority axis 4 ....................................................................................................................... 214
III.5 Priority axis 5: Technical Assistance ................................................................................... 217
PART IV. Reflexions upon the studies in Part I, II and III ............................................................... 219
IV.1 Findings, main conclusions and recommendations ............................................................ 219
IV.2 Summary of the examination of Output Indicators in Part II ............................................... 226
IV.3 Summary of the examination of Result Indicators in Part III ............................................... 250
Appendix to the study on Indicators: Well-developed indicators .................................................... 257
ANNEX B1. List of Meetings held ........................................................................................................ 267
Regional visits ................................................................................................................................. 267
Meetings with central institutions .................................................................................................... 267
Other meetings and similar appearances ....................................................................................... 268
List of people met during meetings ................................................................................................. 268
CENTRAL LEVEL ....................................................................................................................... 268
REGIONAL LEVEL ..................................................................................................................... 271
5
Annex B2. Terms of Reference ........................................................................................................... 273
Annex B3. Generic Question Frame for meetings .............................................................................. 286
Annex B4. Basic Survey on the functionality of the HRD OP: Questionnaire ..................................... 289
Annex B5. Survey on Indicators: Questionnaire.................................................................................. 296
6
Abbreviations
CAO
CFCU
CGMTs
CoHE
ESF
EU
EUD
FAQ
FMU
GDP/c
GVA
HCI
HDI
HRD
HRD OP
IMIS
IPA
IPA IR
IPA MD
IPTA
ISCED
ISCO
İŞKUR
KOSGEB
LE
MIPD
MIS
MoD
MoLSS
MoNE
NACE
NAO
NGOs
NIPAC
NQS
NUTS
OCU
OIS
OP
OS
PES
QA
QACU
RC OP
RGMTT
SMC
SME
SPO
SSI
SWOT
TAT
TNA
Competent Accrediting Officer
Central Finance and Contracts Unit
Central Grant Monitoring Teams
Council of Higher Education
European Social Fund
European Union
Delegation of the European Union to Turkey
Frequently Asked Questions
Financial Management Unit
Gross Domestic Product per capita (inhabitant)
Gross Value Added
Harmonised Consumer Prices (index)
Human Development Indicators
Human Resources Development
Human Resources Development Operational Programme
Integrated Monitoring Information System
Instrument for Pre-Accession Assistance
IPA Implementing Regulation
IPA Management Department
Information, Publicity and Technical Assistance Unit
International Standard Classification of Education
International Standard Classification of Occupations
Turkish Employment Agency
Small and Medium Sized Enterprises Development Organization
Larger Enterprises
Multi-annual Indicative Planning Document
Management Information System
Ministry of Development
Ministry of Labour and Social Security
Ministry of National Education
Statistical classification of economic activities in the European
Community
National Authorising Officer
Non-Governmental Organisations
National IPA Coordinator
National Qualifications System
Nomenclature of Units for Territorial Statistics (Regional Classification)
Operation Coordination Unit
Operation Identification Sheet
Operational Programme
Operating Structure
Public Employment Service
Quality Assurance
Quality Assurance and Control Unit
Regional Competitiveness Operational Programme
Regional Grant Monitoring and Technical Assistance Team
Sectoral Monitoring Committee
Small and Medium Sized Enterprises
State Planning Organization
Social Security Institution
Strengths, Weaknesses, Opportunities, and Threats
Technical Assistance Team
Training Needs Analysis
7
ToR
TURKSTAT
TVET
VET
VQA
Terms of Reference
Turkish Statistical Institute
Technical and Vocational Education and Training
Vocational Education and Training
Vocational Qualifications Authorities
8
1. Executive Summary
This evaluation aims to make an independent assessment of the overall relevance, effectiveness and
efficiency of the implementation of the HRD OP with a view to provide all the stakeholders with reliable
data, analysis and feedback that will help the upcoming OP revision.
During the Interim Evaluation, the Technical Assistance Team has carried out desk studies, has held
more than 52 meetings with key stakeholders at both central and regional level, and has conducted
surveys of semi-qualitative nature. The Interim Evaluation has been based on a combination of
quantitative and qualitative studies with the main focus on the latter.
The examination of the efficiency of the management structure revealed some shortcomings in the
functioning of the HRD OP, mostly in terms of level of knowledge about content and practice as well
as experience. To some extent this reflects the functioning of the SMC, which has limited room for
discussion and merely serves as top-down information.
The examination of the effectiveness and efficiency of the HRD OP indicates that the criteria in the
HRD OP and Calls for Proposals are not completely identical, and that set of criteria is rather broad
and open, which allows the applicants to propose rather different project ideas. The examination
indicates that the chosen approach could thus dissolve the results and outputs in such a way that it
would be difficult in the end to measure common impact.
Overall, the study indicates rather normal distribution of grants per region and per population unit
though the variations are still significant. In case of some regions, variations are still notable. Some
regions receive a lower amount of projects per population unit than others and also the success rate of
applications appears to be lower in these regions. This indicates that the awareness of the grant
schemes is still at a reasonable level but the quality of applications is lower than the quality of
applications in more successful regions. It also appears that due to global competition, the regions
with a strong proactive central institution as well as with strong project writing capacity tend to do
better in the project selection process.
Local absorption capacity appears influenced by the possibilities of potential applicants at the time the
calls for proposals are launched. Although this factor should not cause differences between regions, it
could lower the absorption capacity to a certain extent.
The examination of indicators, their efficiency and relevance, revealed a total of 91 Output indicators.
The examination of the 25 Result Indicators, however, revealed that approximately half of the named
indicators were in fact Output indicators. But even worse, the examination clearly showed that the
HRD OP has no Impact indicators which make ex-post evaluation fruitless if not directly impossible.
The Interim Evaluation finds that the current system is:
•
Insufficient since the current set of indicators lack Impact measures
•
Inappropriate since many indicators have no achievable baselines, lack data, lack
infrastructure, and in certain cases are not well defined.
•
Too ambitious due to the number of indicators
The Interim Evaluation has identified 30 main recommendations. Concerning the efficiency and
effectiveness of the HRD OP management structure:
1. Ensure far more pragmatism and flexibility based upon know-how and knowledge of “content”
instead of narrowly focusing on administrative procedures;
9
2. Ensure stronger commitment and involvement from the Operation Beneficiaries
Concerning the Monitoring Arrangements, notably the Monitoring Information System, MIS:
3. Ensure that the MIS system becomes more user-friendly or at least open up for a dialogue on
how to ensure consistent entering of data to the system
4. Ensure that the system actually reflects the need (the issue of the number and quality of
indicators, which definitely needs to be redefined)
5. Ensure a interlink with basic data, including official statistics and administrative records when
needed for measurement
6. Ensure solid training in quality assurance on all levels, notably on micro level where data are
entered
7. Clean all records in close liaison with caretakers of administrative records, notably SSI,
İŞKUR, and MoNE
8. Make quality controls on a permanent basis.
Concerning the efficiency and effectiveness of the HRD OP contractual, tendering, and financial
procedures:
9. Ensure far more readiness and speed in the evaluation procedures and try to avoid conflicts in
central level responses to implementation level
10. Consider to establish quotas in order to ensure that all regions have at least a limited number
of projects reflecting the needs on the local level.
Concerning the efficiency and effectiveness of the HRD OP Indicators:
11. Revise the current set of indicators. The need for a reform is urgent;
12. Revise the indictors toughly until a decent and doable target is reached. The Interim
Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output,
six Outcome, and six Impact Indicators
13. Understand the basics in evaluation and design a list of indicators which summarizes the main
and basic idea and purpose of the overall programme;
14. Establish a multi-level system of indicators, ensuring that the main evaluation reports only
contains a very small number of smart indicators;
15. Establish solid training programmes in monitoring and evaluation, data control, and quality
assurance, not least on regional level. Place ownership and delegate task where the single
stakeholders can see the purpose.
In case the HRD OP intend to stand up to the words of integration of the horizontal issues of
sustainable development and environmental protection the Interim Evaluation recommends:
16. to integrate impact indicators in the programme for the ex-post evaluation to ensure that the
HRD OP has achieved to ensure sustainable development or least made progress in that
direction
17. to redefine and establish outcome indicators and impact indicators which particularly focus on
environmental protection
18. to ensure that environmental protection is acknowledged as a future demand in the
educational sector notably on VET/TVET level (making it an integral part of new curriculum)
19. to ensure that topics related environmental protection is an integral part of all Lifelong
Learning Trainings underneath Priority 3
20. to give priority to environmental protection topics in priorities and measures addressing gender
equality as well as social inclusion
21. to establish Training Needs Analysis (anywhere in the complete programme) which addresses
the topic of environmental protection both in order to grab a demand in time and partly to
increase awareness of the subject among the employers.
10
The Interim Evaluation has performed a solid investigation on participation. The study reveals
problems with participation on regional level mainly though some problems with the SMC construction
are evident. The study of the participative approach recommends:
22. Support and continue the strengthening of the partnership approach on central level, not least
with respect to the structure of the SMC and its meeting procedures
23. Actively strengthen, perhaps through legislation, a far more viable partnership approach on
regional level.
And the regional level is subsequently examined where the main recommendations of the Interim
Evaluation are:
24. To ensure a solid platform on the regional level with a consolidated and consistent structure,
the build of a professional secretariat and governed by a tripartite body perhaps inspired by
the way the RDAs are constituted
25. Ensure “promoters” (ambassadors) are based regionally and that the promoters are able to
act as support in every phase. The TAT cannot see why the RGMTTs and the promoters are
not directly interlinked or perhaps even the same persons
26. Ensure means of commitment from Social Partners and other main regional actors on a
regional basis by placing ownership through direct involvement in the programming,
monitoring and evaluation phases
27. Ensure far more active involvement from both İŞKUR and RDAs in regional programming,
planning, design of indicators and targets;
28. Design objectives, measures, indicators and targets on regional level and establish a system
of reporting in accordance to a template designed from central level (which could feed directly
in to evaluation reports, like the SAR) to promote and reinforce regional involvement in the
HRD OP
29. Ensure that local members are part of the process (and not just representative from the
Growth centres) to ensure a far more active spread of the programmes internally in the
regions
30. Ensure gender equality, local representation, and representation of target groups in all
advisory boards to the HRD OP.
11
2. Introduction
2.1 Overview of the HRD OP priority structure, measures and objectives
The HRD OP priority structure is as follows:
Priority axis 1: To attract and retain more people in employment, particularly by increasing labour
force participation of women, and decrease unemployment rates especially for young people
Measure 1.1 To promote women's participation into the labour market, and increase female
employment, including those formerly employed in agriculture
Measure 1.2 To increase employment of young people
Measure 1.3 To promote registered employment
Measure 1.4 To improve the quality of public employment services
Specific objective 1: Promote labour force participation and employment of women, including those
formerly employed in agriculture
Specific objective 2: To increase youth employment
Specific objective 3: To promote registered employment
Specific objective 4: To improve public employment services
Priority axis 2: To enhance investment in human capital by increasing the quality of education,
improving the linkage between education and labour market, and raising enrolment rates at all levels
of education, especially for girls
Measure 2.1 To increase enrolment rates particularly for girls with a view to developing female human
resources and access to labour market
Measure 2.2 To improve the content and quality of Vocational and Technical Education
Specific objective 1: To increase enrolment rates particularly for girls at all levels of education and
vocational training
Specific objective 2: To increase the quality of education especially in vocational education and
training
Specific objective 3: To develop innovative approaches to improve linkage between education and
labour market
Specific objective 4: To promote a “partnership approach” to modernize VET system
Priority axis 3: To promote a “partnership approach” to modernize VET system.
Measure 3.1 To promote the development and implementation coherent and comprehensive
strategies for lifelong learning.
Measure 3.2 To increase adaptability of employees and employers by investing more in human
capital.
Specific objective 1: To promote Life Long Learning (LLL) Opportunities under a LLL Strategy.
Specific objective 2: To improve quality of non-formal trainings
Specific objective 3: To increase adaptability of employees
Specific objective 4: To increase adaptability of employers in SMEs
Specific objective 5: To promote well-functioning of the National Qualifications System
Priority axis 4: To promote an inclusive labour market with opportunities for disadvantaged people,
with a view to their sustainable integration into the labour force and combat all forms of discrimination
in the labour market.
Measure 4.1 To increase employability of disadvantaged persons, facilitate their access to labour
market, and eliminate barriers for their entrance into labour market.
12
Measure 4.2 Better functioning and coordination among the institutions and mechanisms in the field of
labour market and social protection particularly in order to facilitate the integration of disadvantaged
persons into the labour market.
Specific objective 1: To facilitate sustainable integration of the disadvantaged into the labour market.
Specific objective 2: To improve the functioning and coordination of institutions and mechanisms in the
field of labour market and social protection, particularly in order to facilitate the integration of
disadvantaged persons into the labour market.
Priority axis 5 for Technical Assistance:
Measure 1 Support for management, implementation, monitoring, control, evaluation and
dissemination activities.
Measure 2 Support for development of absorption capacity of final beneficiaries.
Measure 3 Information and publicity activities.
Specific objective 1: Strengthening the necessary capacity at central level to efficiently develop,
implement, evaluate, monitor and control the IPA funds in the period 2007-2009 within the framework
of HRD OP and improving the information about the ESF structures and best examples, providing
assistance in the transition to Decentralised Management without ex-ante control of the EC
Delegation.
Specific objective 2: Increasing the absorption capacity of the final beneficiaries and administrative
capacity of stakeholders which may get more responsibilities in the upcoming period.
Specific objective 3: Increasing the information and public awareness with respect to the preparation
for and effective use of IPA funds in Turkey in line with the HRD OP.
2.2 Purpose of the Interim Evaluation and evaluation questions
The main objective of this evaluation is to make an independent assessment of the overall relevance,
effectiveness and efficiency of the implementation of the HRD OP with a view to provide all the
stakeholders with reliable data, analysis and feedback that will help the upcoming OP revision
exercise.
The specific objectives of the assignment are:
- to assess the relevance (in particular with the strategic documents such as the Strategic
Coherence Framework , Multi-Annual Indicative Planning Document , employment guidelines,
strategic community guidelines and the Lisbon Strategy), efficiency, effectiveness of the
programme.
- to assess the complementarity between the HRD OP and Regional Competitiveness
Operational Programme (RC OP).
- to provide data on the output and result indicators agreed upon in the HRD OP that cannot be
obtained through monitoring system.
- to assess the achievement of the horizontal principles.
- to identify best practices, factors for success and failure and provide recommendations to
improve the remaining programming and implementation of the HRD OP.
13
Table 1 Evaluation Questions
Q1
Q1.1
Q1.2
Q1.3
Q1.4
Q1.4.1
Q1.4.1.1
Q1.4.2
Q1.4.2.1
Q1.4.2.2
Q1.4.2.3
Q1.4.2.4
Q1.4.3
Q1.4.3.1
Q1.4.3.2
Q1.4.3.3
Q1.4.3.4
Q1.4.3.5
Q1.4.3.6
Q1.4.3.7
Q1.4.3.8
Q1.4.3.9
Q2
Q2.1
Q2.2
Q3
Q3.1
Q4
Q4.1
Q4.2
Q4.3
Q4.4
Q4.5
Relevance, efficiency and effectiveness of the HRD OP
The appropriateness of the management structures formed at the levels of Operation Structure
and Operation Beneficiaries
The use and financial allocation of financial assistance
The quality of projects designed to achieve the objectives, targets and indicators determined in
the HRD OP
Main Analyses
A. Analysis of Previous Evaluation’s Results
Review the results of the ex-ante evaluation
B. Analysis of the Relevance of the HRD OP
Analyse and review the major developments since 2007 at national and EU level that
influence the HRD OP
Analyse the HRD OP’s coherence with the objectives of pre-accession assistance,
strategic documents such as the SCF, Lisbon Strategy, MIPD, employment guidelines,
strategic community guidelines
Make a SWOT analysis of the beneficiaries and stakeholders
Analyse the accuracy, clarity, quality, usefulness and internal consistency of the overall
objectives, purposes, targets, indicators and eligible activities at the priority and measure
level of the HRD OP
C. Analysis of the HRD OP’s Effectiveness and Efficiency
Review programme outputs and results in relation with the expenditures incurred at priority
and measure levels; in the light of financial and physical progress
Collect data on indicators not provided through the regular monitoring system
Make an overall examination of the HRD OP’s programming, monitoring and evaluation
structures and procedures (in particular of the content, timeliness and compliance) to
provide an assessment of the public administration systems’ efficiency and effectiveness
in terms of constructing the necessary mechanisms for attaining the objectives under HRD
OP
Assess the concrete progress of the HRD OP on the basis of the indicators
Examine the efficiency of Operation Selection Criteria
Analyse the role of national bodies, the relations/co-ordination with local authorities,
institutions, beneficiaries and other stakeholders, the commitment of all the stakeholders
Assessment of local absorption and implementation capacities
Analyse the financial management of the budget
Analyse the accuracy of the budget allocations under measures
Complementarity between the HRD OP and RC OP
RC OP’s objectives have been achieved through programming and implementation of the HRD
OP
RC OP has influenced the management structures of the OS and Operation Beneficiaries of
HRD OP
Output and result indicators agreed upon in the HRD OP that cannot be obtained through
monitoring system
Analyse and provide data on output and result indicators agreed upon in the HRD OP that
cannot be obtained through monitoring system
What are the results of the programme in achieving the horizontal principles during the design and
implementation of Operations?
Equal opportunities for men and women
Sustainable development & environmental protection
Participation of civil society
Geographic, sectoral and thematic concentration
Concerns of disadvantaged persons
14
Q4.6
Q5
Q5.1
Good governance
Best practices, factors of success and failure regarding the planning and implementation process
of the HRD OP
Propose recommendations to the actors involved in the programming, determining financial
allocations, monitoring and evaluation
2.3 Outline of the Interim Evaluation Report
Chapter 3 describes the main methods and the approach to the interim evaluation subject to the
evaluation questions just described. The chapter further describes some of the limitations in the study,
including the problems encountered and the way the challenges subsequently were handled.
Chapter 4 subsequently contains the main report on findings, conclusions and recommendations.
Some parts of the study are however viewed as less important for the main report and are
subsequently addressed in annexes. That goes for the Socio-Economic Background Study, the study
of Major Developments on National and EU level, the study of the Ex-ante Evaluation Results, and the
Study on Indicators. The three parts, which basically is part of the set of evaluation questions raised,
can be found in:
 Annex A1 Socio-Economic Study
 Annex A2 Major Developments on National and EU level since 2007
 Annex A3 Results of the ex-ante evaluation
 Annex A4 Annex to the study of Efficiency and Effectiveness
 Annex A5 The study on Indicators
Chapter 5 provides an overview of the complex of annexes to the report.
Finally, as more or less independent parts of the Interim Evaluation Report, two Thematic Reports
have been developed which due to methodological considerations can be difficult to separate from the
rest of the studies:
 Thematic Report I on Complementarity between RC OP and HRD OP
 Thematic Report II on Horizontal Issues
15
3. Main methods
3.1 General approach to the Interim Evaluation
The methodology is based on the best practice procedures in critical academic evaluation of
programmes and templates to which the ToR refer.
The overall methodology and approach to the assignment is based on a high degree of transparency
and stakeholders’ involvement. The approach is based upon international standards of good practices
in evaluation of programmes. Moreover, our approach has focused on making sure that every element
of the studied practices and outcomes are understood and agreed by the stakeholders.
The Interim Evaluation has focused on four types of indicators following good practices as
recommended by EU external programme and as described in Chapter 8. The main purpose of the
evaluation is to evaluate the HRD OP in accordance to the following:

Relevance: The relevance of the programme relates primarily to its design and the extent to
which the stated objectives correctly address the identified problems or real needs. It includes an
assessment of the quality of programme and its design (that is, the logic and its justification in real
issues, and the internal logic and coherence of the programme design).

Efficiency: This assesses how well the various activities (as set out in the logical framework, work
plans and work) transformed the available resources into intended results in terms of quantity,
quality and timeliness. This generally requires comparing alternative approaches to achieving the
same results, to see whether the most efficient process has been adopted.

Effectiveness: The effectiveness of the programme relates to how far the results of the
programme have achieved the actual programme purpose and how assumptions have affected
programme achievements. This will include a specific assessment of the benefits accruing to
target groups, including women and men and identified vulnerable groups such as children, the
elderly and disabled (where appropriate).

Impact: The impact of the programme involves an assessment of the relationship between the
purpose and overall objectives of the programme and the extent to which the benefits received by
the target beneficiaries have had a wider overall effect on larger numbers of people in the region
and the wider policy or sector objectives (as summarised in the programme’s Overall Objective).

Sustainability: This relates to whether the positive outcomes of the programme are likely to
continue after external funding ends, with particular reference to factors of ownership by
beneficiaries, policy support, economic and financial factors, socio-cultural aspects, gender
equality, appropriate technology, environmental aspects, and institutional and management
capacity.
16
3.2 The evaluation methodology subject to the evaluation questions
The Socio-Economic Study in Annex A1
The Report includes a Socio-Economic Analysis based upon a limited number of socio-economic
indicators within the areas of economic, demographic, educational, and labour market indicators. The
purpose is to understand the relevance and the scope of the HRD OP and to evaluate the Operation
Selection Criteria.
The socio-economic background study has drawn on a list of international harmonized indicators,
including not least ETF indicators and indicators for the use of monitoring the European Employment
Strategy. Additional indicators e.g. HDI-indicators (UN) and KILM (ILO) have been tested for relevance
and efficiency in evaluation. However, the Interim Evaluation found that the most obvious indicators
are not available on NUTS 2 level. Nevertheless, the Socio-Economic Study provides insight in data
which can be used as indictors of the HRD OP.
The desk study was performed including an examination of the management information system, MIS,
for the examination of the management structure and effectiveness and efficiency of the HRD OP. The
findings of the desk study were subsequently followed up by a long sequence of meeting with various
actors on central and regional level.
Mapping of the institutional environment and studying current practices
A mapping of the institutional environment in which the HRD OP operates has been conducted during
the very early phase of the implementation. The purpose of the mapping was partly to understand the
institutional and organisational structure of stakeholders in the HRD OP, and partly to ensure a solid
background for the evaluation of the manage structures in HRD OP.
The mapping was used for the examination of roles and the functioning of the management structure
of the HRD OP.
In the evaluation of current practices under evaluation question 5 the Consultant primarily focused on
the conception of the various stakeholders. The Consultant has substantial experience in involvement
of stakeholders in the process. The Consultant especially focused on evaluation of practices and
outcomes of implementing the OP.
In that respect, bilateral meetings with the stakeholders has ensured solid feedback on the perception
of practices. Where practices including logistics and organisation of working procedures appear
impractical or less feasible the Interim Report has proposed recommendations for improving the
system.
Accordingly, the study of current practices is based upon a quantitative approach with a high degree of
self-evaluation and self-reflexion. A question frame based upon standardized questions directed to all
target institutions, beneficiaries and stakeholders was designed in order to evaluate relative weakness
and strengths.
During the complete examination of roles, structure and procedures the Interim Evaluation has
explicitly assessed the involvement of external stakeholders, both employers and employees, in the
HRD OP. The main findings and observations made are mainly addressed in the Thematic Report on
Horizontal Issues.
17
Finally, a survey was conducted in the very end of the project, providing further support to the findings
from the examination.
The logic of the Report and the way the evaluation question was addressed
The logic of the Interim Evaluation Report is fundamentally based upon the assessment structure
followed by the evaluation questions. The number of evaluation questions is fairly impressive and so is
the underlined tasks connected to each of the evaluation questions. This section briefly introduce the
way the evaluation questions has been handled, assessed, and approached in the report.
Q1 Relevance, efficiency and effectiveness of the HRD OP
Q1.1 The appropriateness of the management structures formed at the levels of Operation
Structure and Operation Beneficiaries
The main approach to the study of the is obviously of qualitative nature where the main point is to how
well the current structure is designed to cope with the both the administrative tasks related to running
a large scale HRD OP as well as how the management structure has the natural ability to cope with
the technical and methodological problems connected with actions within the area of Human Resource
Development.
In general, the organisation of HRD strategies is a complex matter somehow illustrated by figure 1.
18
Figure 1. The organisation of Human Resource Development Strategies and the compliance
with other prime strategic policy areas
The figure shows how HRD strategies involve a number of core policy areas, which all have to be
taken into account. And yet again, the figure is only illustrative in terms of displaying the interaction
with other main policy areas; obviously it does not illustrate the need for alignment of policies and
hence the need for coordination and communication between the core policy areas (and often
Ministries and their administrative bodies). But more so, the figure only stipulates how the various
geographical levels have to be taken into account: It appears obvious that it tend to be a complete
waste of means to implement actions where the need is only marginal. And finally, the ways to
incorporate the views and opinions of the stakeholders involved in HRD is hardly reached in the figure,
though still being of vital importance for the ability of the strategy to address the needs and to achieve
the goals of the strategy.
In the assessment of the relevance of the management structure of the HRD OP in Turkey the Interim
Evaluation has focused on the following:
 The extent the objectives of the HRD OP are in compliance with the overall national agenda in
Annex A2
 The extent the objectives of the HRD OP are mirrored in the agenda of line ministries in Annex
A2
 The extent the objectives of the HRD OP comply with the national agenda as described in
Annex A2
 The extent the objectives of the HRD OP comply with the overall agenda and strategies in the
EU as assessed in Annex A2
 The extent the objectives of the HRD OP are mirrored in the Indicators and their targets as
assessed in Annex A5
19







The extent the administration of the HRD OP are taken on board by the management
structure as assessed in Chapter 4 and Annex A5
The extent the administration of the HRD OP are able to direct the incentives and actions into
a HRD Framework as assessed in Chapter 4, which obviously includes the ability to make
solid follow through monitoring arrangements (Chapter 4) and evaluation as assessed in
Annex A5
The extent the Social Partners and similar private and public stakeholders are taken into
account as assessed in Thematic Report II
The extent the Social Partners are ready to take on board the responsibilities in a participative
approach as assessed in Thematic Report II
The extent the overall structure within the governmental institutions involved in HRD strategies
are ready to take on board the responsibilities to manage, design policies and implement
actions within the area of HRD as assessed in Chapter 4
The extent the central administration approach the partnership approach as assessed in
Chapter 4 and Thematic Report II
The extent the regional and local level needs and priorities are taken into account in the
overall framework, the objectives, the management (on consultancy basis), and in
identification of indicators and their targets as assessed in Thematic Report II, Chapter 4 and
Annex A5
In addition, the assessment includes a SWOT on the main stakeholders (see below).
The methods used are a mix of quantitative and qualitative studies based upon desk studies, studies
of data from the central Monitoring System, the national statistical office (TÜİK/Turkstat), and
administrative records (unfortunately primarily only from İŞKUR, the Turkish Public Employment
Service), and interviews with key persons in both regional and central institutions.
In addition a survey was conducted to collect structure viewpoints and opinions on the overall
management structure of the HRD OP as well as its relevance. The results from the survey had only
been used to support the findings in the evaluation studies undertaken through other means and
methods; the survey was not designed to stand alone insofar the main point was to obtain some very
fragmented viewpoints including providing a method about the knowledge of the HRD OP among the
stakeholders.
A total of 52 meetings were held during the preparation of the Interim Evaluation Report, including 26
meetings on regional level and 26 meeting with key stakeholder institutions on central level. The
meetings were conducted according to a pre-prepared question frame (see Annex B7 as well as the
methodological sections in the studies).
Q1.2 The use and financial allocation of financial assistance
Q1.4.3 C. Analysis of the HRD OP’s Effectiveness and Efficiency
Q1.4.3.1 Review programme outputs and results in relation with the expenditures incurred at
priority and measure levels; in the light of financial and physical progress
Q1.4.3.5 Examine the efficiency of Operation Selection Criteria
Q1.4.3.7 Assessment of local absorption and implementation capacities
Q1.4.3.8 Analyse the financial management of the budget
Q1.4.3.9 Analyse the accuracy of the budget allocations under measures
All the evaluation questions related to procedures, financial allocation, expenditures, local absorption
and implementing capacities, accuracy and financial management are gathered in the examination of
the HRD OP’s Effectiveness and Efficiency. The main judgement criteria for these entrances to the
20
study of the effectiveness of the HRD OP follows best practices and should otherwise be evident from
the context, since the main point is to assure effectiveness in the programme.
Q1.3 The quality of projects designed to achieve the objectives, targets and indicators
determined in the HRD OP
The Interim Evaluation has only limited means to undertake a full study of the quality of the projects
within the complete HRD OP for obvious reasons, notably the allocated time. Thus the main focus is to
evaluate the relevance of criteria for selection as well as the criteria for follow up in terms of output,
outcome, and impact of the operations and measures taken. The evaluation method is further
supported by the overall delays in the complete programme, which has had created delays in the
implementation of the single projects, and thus created limited means to study the overall impact of
the measures taken, and thus the projects assigned to meet the purposes of the objectives.
The Interim Evaluation has subsequently made use of a mix of methods to evaluate the quality of the
projects (or perhaps rather the measures and actions taken) based upon the following:
 An assessment of regional and local absorption in Chapter 4
 An assessment of the regional and local stakeholders’ ability to have impact on the projects
implemented in their region in Thematic Report II
 An assessment of the selection criteria on an overall level in Chapter 4
 An assessment of the Output indicators to monitor the projects in Annex A5
 An assessment of the Result indicators to monitor and evaluate the projects and their impact
in Annex A5
Q1.4.1 A. Analysis of Previous Evaluation’s Results
Q1.4.1.1 Review the results of the ex-ante evaluation
A study of the follow-up on the ex-ante comments and recommendations has been undertaken
throughout the Interim Evaluation studies. Basically, the Interim Evaluation has no finer criteria and
method underlined the follow-up except from the traditional examination based upon 1) Desk studies;
2) Interviews with prime stakeholders on central and regional level; and 3) the examination of the
relevance of priorities, measures, objectives, targets and indicators which follows from the general
examination of the relevance of the HRD OP. The main results of the study can be found in Annex A3
(though parts of the study are reported into the studies elsewhere in the Interim Evaluation Report).
Q1.4.2.1 Analyse and review the major developments since 2007 at national and EU level that
influence the HRD OP
Q1.4.2.2 Analyse the HRD OP’s coherence with the objectives of pre-accession assistance,
strategic documents such as the SCF, Lisbon Strategy, MIPD, employment guidelines,
strategic community guidelines
The study on Major Developments since 2007 follows no general practical overall method per se. The
general approach is plain desk study and can be found in Annex A2 to the Interim Evaluation Report.
Q1.4.3.6 Analyse the role of national bodies, the relations/co-ordination with local authorities,
institutions, beneficiaries and other stakeholders, the commitment of all the stakeholders
Q1.4.2.3 Make a SWOT analysis of the beneficiaries and stakeholders
The making of SWOT analysis is definitely not done lightly. All SWOT analyses suffer from a high
degree of subjectivity and cannot be used as background for scientific studies without an extremely
21
clear methodology, which basically eliminate the complete structure of the SWOT insofar the SWOT
analysis always serves to the purpose of making clear statements based mainly on a high degree of
self-assessment. The nature of SWOT is subjective whether performed by an independent observer or
done as a self-assessment exercise among the involved stakeholders.
The main approach to the SWOT conducted in this report is based upon a subjective assessment of
the criteria mention under Q1.1 above; which means the ability to perform and conduct the
management of a HRD strategy based upon observation and reflexions obtained either from own
observations through meetings with the examined organisations and from other stakeholders within
the programme; normally called a 360 degrees assessment.
The meetings with the stakeholders, including the OS Units, followed more or less the same structure
and were based upon question frames which included question within the following brackets:
 Function of the unit and job functions
 Responsibilities
 Procedures for follow-up
 Work flows
 Problem areas, including:
o Understaffing
o Poor coordination (which includes not least ability to perform Social Dialogue)
o Lack of legal basis (how does that influence your work)
o Ownership
o Involvement with external stakeholders
The Interim Evaluation TAT, however, accentuates that the SWOT in this Interim Evaluation Report
cannot stand alone since it refers and relates to the studies in other parts of the report.
Q1.4.2.4 Analyse the accuracy, clarity, quality, usefulness and internal consistency of the
overall objectives, purposes, targets, indicators and eligible activities at the priority and
measure level of the HRD OP
The evaluation question draws upon various activities and not least competences which cannot be
gathered in one single approach. In general, the main part follows the approach mentioned under
Q1.1 above, but in addition the study related to procedures, financial allocation, expenditures, local
absorption and implementing capacities, accuracy and financial management gathered in the
examination of the HRD OP’s Effectiveness and Efficiency above as well as (and not least) the study
on indicators mentioned below.
As for the Interim Evaluation, the study on the relevance of the chosen indicators in Annex A5
combined with the study on the readiness of the management structure and the study of Effectiveness
and Efficiency both in Chapter 4 address the highly complex matters mentioned in the evaluation
question.
The study on Indicators also constitutes a study of relevance of the objectives in combination with the
Socio-Economic Study in Annex A1.
Q1.4.3.2 Collect data on indicators not provided through the regular monitoring system
Q1.4.3.3 Make an overall examination of the HRD OP’s programming, monitoring and
evaluation structures and procedures (in particular of the content, timeliness and compliance)
22
to provide an assessment of the public administration systems’ efficiency and effectiveness in
terms of constructing the necessary mechanisms for attaining the objectives under HRD OP
Q1.4.3.4 Assess the concrete progress of the HRD OP on the basis of the indicators
Q3 Output and result indicators agreed upon in the HRD OP that cannot be obtained through
monitoring system
Q3.1 Analyse and provide data on output and result indicators agreed upon in the HRD OP that
cannot be obtained through monitoring system
The study of the indicators was based upon a combined quantitative and qualitative approach since
the study soon revealed serious shortcomings in the design of indicators. The study of indicators was
subsequently focused to explain the fundamental principles in establishing performance indicators and
targets based upon a solid examination of each of the indicators proposed by the HRD OP.
With respect to output and result indicators agreed upon in the HRD OP that cannot be obtained
through monitoring system, the consultant has screened all available data obtained from the operating
system as well as the data supplied from the main actors.
Where data on indicators were not available, the Interim Evaluation has made a note on the lack of
data / information and proposed ways to obtain the data / information in the future (methodology) in
accordance to the methods described in the Inception Report. When it was impossible to obtain the
required data / information or in case the data are inconsistent, misleading, or not relevant for the
measures new alternative indicators have been proposed.
The study of the Result and Output indicators includes:
•
Screening of data and information streams, including accessibility and production dates
•
Study of baselines and their relevance for the time period of HRD OP
•
Study of consistency and validity of data and information, which includes collection
methods
•
Irregularities in deliveries of data and information
•
The relevance of the pre-selected measures and indicators as measures of medium and
long term outcome and impact assessments
•
And in addition the TAT will obtain information and comments to the measures and
indicators from beneficiaries and stakeholders in the programme
•
Analysis of data provided mainly in order to evaluate relevance
•
Propose alternative ways to collect data and information where data and information are
lacking and not available in the nearest future OR proposed new indicators which have
relevance and are accessible.
An important part of the evaluation of indicators was to evaluate the process of producing high quality
data and information available for future monitoring and evaluation as well as programming process.
In that respect, the Interim Evaluation has found it necessary to accentuate the need for ensuring
consolidated data.
A separate introductory part of the Annex A5 on the study of indicators within the Interim Evaluation
Report is almost exclusively devoted theoretical aspects in the respect, simply because the current
system of indicators needs to be improved.
Obviously, the HRD OP needs to be evaluated in accordance to an international set of practices
allowing the Turkish stakeholders to make efficient use of international benchmarking. The Interim
Evaluation has accordingly tried to give some hints to how to do effective monitoring and evaluation.
23
Q2 Complementarity between the HRD OP and RC OP
Q2.1 RC OP’s objectives have been achieved through programming and implementation of the
HRD OP
Q2.2 RC OP has influenced the management structures of the OS and Operation Beneficiaries
of HRD OP
In order to understand how well the two programmes complement each other in Turkey the Interim
Evaluation has made a solid desk study of the main reports, including the two programmes, the
progress reports, the Strategic Coherence Framework, SCF, and the final report of the ex-ante
evaluation. Based upon the desk research the Interim Evaluation has conducted interviews with main
stakeholders on central and regional level, including the contractors and the two main ministries
involved in the management of the programme. And finally a survey was conducted where some few
questions was designed to get viewpoints on the complementarity between the two programmes.
The feedback on potential problems and conflicts were sparse and hardly worth mention.
Nevertheless, the Interim Evaluation has gathered and reflected the main viewpoints in the Thematic
Report I on Complementarity between the RC OP and the HRD OP.
Q4 What are the results of the programme in achieving the horizontal principles during the
design and implementation of Operations?
Q4.1 Equal opportunities for men and women
Q4.2 Sustainable development & environmental protection
Q4.3 Participation of civil society
Q4.4 Geographic, sectoral and thematic concentration
Q4.5 Concerns of disadvantaged persons
Q4.6 Good governance
The study of the achievements in meeting the horizontal principles are based upon a variety of
studies, based upon both quantitative and qualitative methods, notably following the methods
mentioned under Q1.1 above, and further heavily supported by interviews with stakeholders, the study
on indicators (which basically examine how well the principles are integrated in the programme), desk
study on key national documents, and lightly supported by the basic survey mentioned elsewhere
above.
The findings, conclusion and recommendations from the intensive studies are gathered and presented
as main findings, conclusion and recommendations in the Thematic Report II on Horizontal Issues
which further presents a description of the methods used to examine the principles where the methods
differ from the overall methodological approach of the evaluation.
Q5 Best practices, factors of success and failure regarding the planning and implementation
process of the HRD OP
Q5.1 Propose recommendations to the actors involved in the programming, determining
financial allocations, monitoring and evaluation
The study of Best practices within the Interim Evaluation Report has followed two main routes: 1)
international best practices have been used mainly as an intellectual reference throughout the study
and incorporated as some underlined standard where it made sense for the evaluators; and 2) best
practices observed from meetings around the country.
With respect to international best practices the practices has been used as reference points in the
study of Efficiency and Effectiveness and in the study of the Management Structure both in Chapter 4.
But more so international best practices have been heavily drawn upon in the study on Indicators in
Annex A5 and in the examination of the functionality of the MIS in Chapter 4.
24
The Interim Evaluation Report, fully acknowledging the special conditions of Turkey, the largest
country doing HRD OP and experiencing (in EU context) a variety of cultural traditions still completely
not familiar to the EU context, procedures and agendas, has accordingly only drawn upon international
best practices where there was a solid pragmatic and intellectual reason to do so and yet still in
accordance to feasibility given the working habits and working traditions which applies; that is in the
theoretical context of evaluation theory which forms the main approach to the study of indicators and
in the routes to make electronic MIS-systems work in practice1. Furthermore, international best
practices have further been used as a reference for providing recommendations to participation and
involvement of civil society in Thematic Report II on horizontal principles.
When it comes to best practices from Turkey, which the Interim Evaluation team perhaps think is
slightly more relevant for the Turkish stakeholders as reference points, the Interim Evaluation Report
primarily base its recommendation upon observed practices through meetings with key stakeholders
on regional and central level. In that respect, the Interim Evaluation report has benefitted to a large
extent from the relative high number of meetings conducted 2.
Observation of best practices are inserted where appropriate in the context of studies, though not
necessarily explicitly (international best practices) unless the Interim Evaluation found reasons to use
the best practices observed as a reference point for the Turkish stakeholders (notably in Thematic
Report II on Horizontal Issues and almost exclusively with reference to best practices from Turkey).
3.3 Challenges and solutions found
Throughout the Interim Evaluation the Technical Assistance Team (TAT) has faced several
unforeseen challenges. The TAT has done its upmost to meet the challenges and through a high
degree of flexibility, not least in the timing of crucial events, the TAT has been able to find doable
solutions.
Problems experienced
Long inception phase
Postponed meetings on
central level




Postponed meetings at
regional level
Postponed surveys




Difficulties in gathering
basic data


Impact
Difficulties in starting operation
Postponement of meetings
Postponement of surveys
Postponement of meetings at
regional level
Postponement of surveys
Postponed surveys
Postponed follow-up
Postponed analysis phase and report
phase
The way they were handled
The consultant started operation
before approval of the Inception
Report
Rescheduling of work plans
Hampered Data quality examination
Hindered evaluation of relevance and
availability of indicators
Examination of the Indicators was
conducted on a theoretical basis
mainly with heavy inclusion of best
practices and drawing on
experience form impact studies in
general
Rescheduling of work plans
The survey was done during the
reporting phase
1
The team has in particular benefitted from a newly conducted scientific research study for the TVET in Egypt on
International Best Practices in establishing MIS, Knowledge Systems for TVET, LMIS (Labour Market Information
Systems), and CGS (Carrier Guidance Systems) not published yet conducted by one of the members of the TAT
in the evaluation of the MIS.
2 relative to the requirements of the TOR, which requirements the TAT found far too low notably to assess best
practices
25
4. Main findings, conclusions and recommendations
4.1 Examination of the Management and control structures of the HRD OP and
SWOT
The structures and authorities as well as their functions and responsibilities are described in the HRD
OP3. They are in compliance with the relevant provisions of the IPA Implementing Regulation (Articles
21 to 31).
Under the management and control provisions of the Regulation and Article 6 of the Framework
Agreement, the following structures and authorities are designated as management structures and
authorities:
 National IPA Coordinator
 Strategic Coordinator
 Competent Accrediting Officer
 National Authorising Officer
 National Fund
 Audit Authority
 Operating Structures per component or programme to deal with the management and
implementation of assistance under the IPA Regulation;
With the exception of the Operating Structure, these bodies essentially perform tasks that are
generally applicable to all IPA components; their functions are specified in the relevant articles of the
IPA Implementing Regulation and in Annex A of the Framework Agreement concluded between the
Commission and Turkey.
The meetings with the stakeholders, including the OS Units, followed more or less the same structure
and were based upon question frames which included question within the following brackets:
 Function of the unit and job functions
 Responsibilities
 Procedures for follow-up
 Work flows
 Problem areas, including:
o Understaffing
o Poor coordination
o Lack of legal basis (how does that influence your work)
o Ownership
o Involvement with external stakeholders
The SWOT tables mention some of the main observations from a very broad scale.
3
Human Resource Development Operational Programme, second version
26
4.1.1 Operating Structure (OS) 4
The HRD OP describes that the operating structure shall be responsible for managing and
implementing the IPA programme or programmes concerned in accordance with the principle of sound
financial management. The Ministry of Labour and Social Security is designated as Operating
Structure for the HRD Component by the Prime Ministry Circular no: 2009/18, and will be managed by
the Ministry of Labour and Social Security, which in compliance with the Annex A of the Framework
Agreement carries out the following functions:













drafting the annual or multi-annual programmes;
monitoring programme implementation and guiding the work of the sectoral monitoring
committee as defined in Article 36(2) of this Framework Agreement and in Article 59 of the
IPA Implementing Regulation, notably by providing the documents necessary for
monitoring the quality of implementation of the programmes;
drawing up the sectoral annual and final implementation reports defined in Article 38(1)
and (2) of this Framework Agreement and in Article 61(1) of the IPA Implementing
Regulation and, after their examination by the sectoral monitoring committee, submitting
them to the Commission the NIPAC and the NAO;
ensuring that operations are selected for funding and approved in accordance with the
criteria and mechanisms applicable to the programmes, and that they comply with the
relevant Community and national rules;
setting up procedures to ensure the retention of all documents regarding expenditure and
audits required to ensure an adequate audit trail;
arranging for tendering procedures, grant award procedures, the ensuing contracting, and
making payments to, and recovery from, the final Beneficiary;
ensuring that all bodies involved in the implementation of operations maintain a separate
accounting system or a separate accounting codification;
ensuring that the NF and the NAO receive all necessary information on the procedures
and verifications carried out in relation to expenditure;
setting up, maintaining and updating the reporting and information system;
carrying out verifications to ensure that the expenditure declared has actually been
incurred in accordance with the applicable rules, the products or services have been
delivered in accordance with the approval decision, and the payment requests by the final
Beneficiary are correct: These verifications shall cover administrative, financial, technical
and physical aspects of operations, as appropriate;
ensuring internal audit of its different constituting bodies;
ensuring irregularity reporting;
ensuring compliance with the information and publicity requirements.
In addition to the above-mentioned responsibilities resulting from the Annex A of the Framework
Agreement, the Operating Structure will also be responsible for:
 managing the Secretariat of the HRD OP Monitoring Committee;
 ensuring the evaluations of the HRD OP;
4
Please note that the EUD, the CFCU, and to some extent the National Fund, National Authorizing Officer, and
the National IPA Coordinator, all form an integral part of the Operating Structure of the HRD OP. However, the
two (or four) institutions are not explicitly examined in the section, mainly since their functions and roles for the
HRD OP are on a very aggregated level and/or since their functions and roles only are marginally connected to
HRD OP alone. Accordingly, a full examination based upon a 360 degree study of their management structure
would have only very limited value. The named institutions are accordingly not explicitly examined in the following
sections.
27
 implementing the measures under Technical Assistance of the HRD OP.
The HRD Operating Structure is according to the HRD OP composed of the following bodies:
1. IPA Management Department
1.1.
Programming Unit
1.2.
Monitoring and Evaluation Unit
1.3.
Information, Publicity and Technical Assistance Unit
1.4.
Technical Implementation Unit
1.5.
Quality Assurance and Control Unit
2. Internal Audit
3. HRD FCU5
Since the HRD OP was described the composition of the Units has been altered. The structure is now
as follows:
1. The Programming Unit, constituting a staff of 11, has been divided into three sections:
a. Programme Management Section,
b. Procurement Section, and
c. Tender Evaluation and Contracting Section.
2. Financial Management Unit (FMU) is composed of 3 sections and 11 staff
3. Monitoring and Evaluation Unit (M&E) has conducted its activities with 4 staff
4. The Information, Publicity and Technical Assistance Unit (IPTA) have a staff of 5 persons
5. Contract Management Unit has 11 staff
6. The Quality Assurance and Control Unit (QACU) is performing its duties with 5 staff members
7. The Administrative Office has 8 staff
The strong sides of the OS are the young and talented teams in all units and their seemingly ability to
inspire each other. No doubt that they are skilful and well-educated. The Interim Evaluation has made
the following comments on the work of the OS inserted directly in a SWOT study.
SWOT assessment of the Operating Structure (all units)
Strengths

Young team with will-power and
encouragement

Strong will to do everything “by the book” and
in accordance to regulations

Professionalism in approach to the tasks
Weaknesses

Less experience in HRD (content)

The young talent and the will power to prove
generates a stiff and rigid discipline

Lack of pragmatism makes compromises
difficult6
5
In terms of the Cooperation Agreement on Delegation of Certain Tasks of Operating Structure to the Central
Finance and Contracts Unit for the Transition Period, all procedural aspects of the tendering process, contracting
matters and financial management including payment of project activities are delegated to the CFCU, under the
IPA component IV for a transition period of 2007-2010. After the transition period, a Finance and Contracts Unit
for the HRD OP in the MoLSS will be established.
6 “Pragmatism is a philosophical tradition centred on the linking of practice and theory. It describes a process
where theory is extracted from practice, and applied back to practice to form what is called intelligent practice.
Pragmatism is the philosophical attitude that the validity of an idea lies in its practical consequences. Pragmatism
is the first American philosophy developed independently of European schools of thought. Pragmatists have
agreed with traditional empiricists that ideas must be tested against experience. However, they have departed
from the empirical stress on the origin of ideas in experience in order to lay a new stress on the effect of ideas in
experience.” [Quotes from Wikipedia]
Pragmatism is the ability to make things work in practice where scholared books and procedures may insinuate
more pure and straightforward measures. A certain degree of Pragmatism is notably necessary to make actions
where private stakeholders, notably the Social Partners, simply know better how to create an effect on certain
policy measures (or where the lack of involvement from Social Partners may create distortions on the labour
market), which is evidently necessary in every measure related to Human Capital formation. It further stipulates
that compromises are the only way forward.
In this case, the Interim Evaluation insinuate that the OS perhaps tries to be slightly too ambitious in terms of
pursuing (and pushing forward) certain goals, objectives, measures and target, making the OS slightly blind for
the need for consensus about the measures, the actions, the targets, the target groups, and perhaps notably how
28

Knowledgeable in procedures



Opportunities

The accreditation and the forthcoming ability
to govern and manage things more
independently may form a more flexible
platform

With age and not least experience a more
pragmatic and flexible attitude can be created
The handling of the regional dimension
appears weak and founded on principles of
establishing a unified system (the lack of
cooperation between central and regional
levels appears visible)
Lack of knowledge in how to ensure dialogue
between various stakeholders, including
private stakeholders and Social Partners on
local, regional and national level in order to
ensure “content”
Seemingly lack of knowledge on labour
market and educational issues (the OS works
as administrators, not experts on the subject)
Threats

Lack of pragmatism can easily halter the
Social Dialogue and create severe practical
problems for the regional and local level
actors

The focus on doing everything correct can
impose too many restrictions to the local
implementation level, where flexibility is
needed to obtain results in HRD

Lack of legal basis
In general, the Units lack experience from practice. The hardworking units consist of young highly
talented and well-educated persons, perhaps even ambitious, somehow eager to do everything “by
the book”. The feature is of course appreciable but not entirely flexible and do hardly stand a change
when dealing with HRD issues on a practical level which necessitate involvement of external
stakeholders to a high extent, including social partners.
From the Interim Evaluation point of view it appears quite evident from the overall structure of the HRD
OP that the OS mainly maintain the duties as general administrators, which means ensuring that the
monitoring mechanisms are in place, that the stakeholders, including the external stakeholders, are
taken into account, that the procedures, including the PRAG, are followed, that the selection of project
are done in compliance with the regulations, and that the monitoring reports, including the MIS, works
efficiently. That is obviously very appreciable.
However, and at the same time, the Interim Evaluation had difficulties in being assured that the OS
level also possesses the methodological and theoretical expertise and know-how in labour market
issues, educational issues, and social issues, to ensure solid feedback to the central key institutions
involved in the programme. The last thing obviously also necessitate know-how in labour market
analysis, Training Needs Analysis, and similar targeted surveys of labour market demand and training
needs, know-how in design of Active Labour Market Measures and Education Actions for demanddriven education and training, as well as know-how in how to organise actions on local, regional and
central level. This know-how could easily turn out to be of paramount importance insofar the main bulk
of the assessment of the relevance of the content in the programme as well as in single projects
appear to be left to the Operation Beneficiaries almost solely7.
In other words, the core competences within the OS are within general administration indicating that
the methodological and theoretical expertise and know-how in labour market issues, educational
to address the target groups. Accordingly, Pragmatism as a weakness and a threat simply reflects the point that
the OS constitute of general administrators with perhaps only limited experience from practice and negotiation.
7 The point was also addressed several by İŞKUR, the Public Employment Service of Turkey, on both central and
regional level.
29
issues, social issues, and similar have been brought in from outside, including the MoLSS, which
makes the OS units slightly narrow and vulnerable in their entire focus on procedures and
administration. The dialogue is hardly achievable when it comes to content matters since the level of
expertise is not founded and not generated from experience. But then again, perhaps it is not needed
to administrate a HRD OP.
The assessment of the efficiency and effectiveness of the Operating Structure is thus that the
Operating Structure has means to pursuit administrative procedures and ensure a stringent discipline
insofar the units obtain the accreditation and get experience with their new roles thereafter.
The more open question is how the lack of experience and knowledge of the content within the HRD
areas will impact on the effects of the programme, not least sustainability. In that respect the lack of
knowledge in establishing indicators from theory and practice is definitely an important feature when it
comes to on-board the responsibility of defining measures and indicators which are workable in
practice. Thus, and again, the study of indicators appears to be an indicator of the ability of the
Operating Structure to be front runners in the HRD OP, perhaps in particular on the content.
4.1.2 The operation beneficiaries
The operation beneficiaries have the control of the implementation of the operations within the HRD
OP. The operation beneficiaries are Turkish Employment Agency (İŞKUR), Social Security Institution
(SSI), MoLSS, Ministry of National Education (MoNE), The Union of Chambers and Commodity
Exchanges of Turkey (TOBB), and Ministry of Culture and Tourism.
The following table figures out the institutional and financial overview of the Operations being
implemented under HRD OP:
Measure
M.1.1
M.1.2
M.1.3
M.1.4
Title of the Operation
Budget
Operation Beneficiary
Date of
Signature of the
Operational
Agreement
Promoting Women’s Employment- I
27 MEUR
Turkish Employment
Agency (İŞKUR)
8th October 2009
Promoting Women’s Employment- II
30 MEUR
-
Promoting Youth Employment- I
29 MEUR
Turkish Employment
Agency (İŞKUR)
Turkish Employment
Agency (İŞKUR)
Promoting Youth Employment- II
26 MEUR
-
Promoting Registered Employment
Through Innovative Measures- I
13 MEUR
Turkish Employment
Agency (İŞKUR)
Social Security
Institution
(SSI)
Promoting Registered Employment
Through Innovative Measures- II
7 MEUR
-
Improving the Quality of Public
Employment Services- I
12 MEUR
Social Security
Institution
(SSI)
Turkish Employment
Agency (İŞKUR)
Improving the Quality of Public
Employment Services- II
4 MEUR
MoLSS
-
21st December
2009
8th October 2009
21st December
2009
30
Measure
M.2.1
M.2.2
M.3.1
M.3.2
M.4.1
TOTAL
Title of the Operation
Budget
Operation Beneficiary
Date of
Signature of the
Operational
Agreement
Increasing Enrolment Rates
Especially for Girls- I
16 MEUR
Ministry of National
Education (MoNE)
21st December
2009
Increasing Enrolment Rates
Especially for Girls- II
Improving the Quality of Vocational
Education and Training in Turkey- I
10 MEUR
Ministry of National
Education (MoNE)
Ministry of National
Education (MoNE)
-
Improving the Quality of Vocational
Education and Training in Turkey- II
Promoting Lifelong Learning-I
24 MEUR
Ministry of National
Education (MoNE)
Ministry of National
Education (MoNE)
-
Promoting Lifelong Learning- II
15 MEUR
-
Increasing Adaptability of
Employers and Employees to the
Changes in Global Economy- I
5 MEUR
Ministry of National
Education (MoNE)
The Union of Chambers
and Commodity
Exchanges of Turkey
(TOBB)
Increasing Adaptability of
Employers and Employees to the
Changes in Global EconomyTourism
10 MEUR
Ministry of Culture and
Tourism
-
Increasing Adaptability in the Field
of Occupational Health and Safety
Promoting Active Inclusion in
Turkey
4 MEUR
MoLSS
24 MEUR
Turkish Employment
Agency (İŞKUR)
20 MEUR
15 MEUR
-
8th October 2009
-
-
292 MEUR
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme 2010,
pp. 17-19
SWOT assessment of Operation Beneficiary: İŞKUR; the Turkish Public Employment Service
Strengths

Solidly founded organisation with strong
commitment on both central and regional level

Strong expertise in analysis and handling of
data

Strong capacity in handling and management
of projects

High degree of professionalism in labourmarket related issues within the context of
HRD OP

Commitment but not without flexibility
(experience with practical issues and
challenges in HRD)

Strong will to listen and become engaged in
new processes

Regional level has become much stronger
and more active the past five years

Stronger focus on Active measures for a
longer period has ensure readiness to deal
with HRD issues

Solid data provider
Weaknesses

Heavy organisation and perhaps with some
long communication lines

Too little involved in the implementation and in
monitoring the HRD OP on regional level8

Still no uniform approach to barometer
surveys on labour market demand and
shortages

Still slightly weak on analytical expertise on
regional level (stronger cooperation with RDA
appears to be encouraged, though; but still
the RDAs expertise on these matters of labour
market analyses appears an open question)
There are 19 RGMTs. At the beginning, İŞKUR was asked to involve at least two experts in each RGMT but this
was not done with the justification of workload of regional directorate.
8
31

Solid partner as CGMT
Opportunities

In relation to HRD OP: Not relevant
Threats

In relation to HRD OP: Not relevant
Both İŞKUR and MoNE are mature organisations with solid experience in project making, including
donor funded projects. The SWOT analysis clearly illustrates the point. And this is where the
similarities end. The nature of the two organisations determines their strengths but also the
weaknesses. Not that it’s a bad sign – it is highly common – the issue is just a matter of how well the
two organisations are able to make use of each other strengths for the benefit of the programme, and
Turkey of course.
SWOT assessment of Operation Beneficiary: MoNE, The Ministry of National Education
Strengths

Solid knowledge of project implementation

Strong CGMT

Solid knowledge on educational issues

Massive organisation with huge possibilities to
actively make a demand driven impact
Weaknesses

Heavy and difficult manageable administration
with many different entities, each with their
own highly differentiated expertise

Too narrow focus on education in relation to
HRD OP, mainly on topics concerning
outcome

Far too weak in labour oriented themes,
including issues of “demand driven education”

Statistical knowledge is still rather weak; the
need for solid data appears not to be
recognized

No uniform approach to TNA and surveys on
Labour Market Imbalances and Shortages
Opportunities

More bridge building between VET/TVET and
HE

Potentials for design of new VET/TVET
courses with large know-how based on
research

Research founded VET educations makes the
inclusion of issues of environmental protection
potentially more demand driven

Potentials for innovative educations which
stimulates and reinforce the cooperation
between the educational sector and private
enterprises (inducing larger economic growth
potentials)
Threats

As in many other countries the stringent focus
on educational issues, including management
of education and training institutions, can
make the education and training institutions
blind for employment effects

The Universities plays an important role in
VET/TVET; it might give rise to:
o Less labour market related
educations
o Weak sustainability in TVET sector
due to the natural focus on HE (HE is
the fundament, the platform,
including the financial basis, for the
universities obviously)
o Sustainability in TVET may be
questioned as soon as the donor
funded projects has ended (the
economy is anchored in HE)
o And sustainability in TVET may be
threaten if the economy slows down
and goes into recession
From an outside perspective on behalf of the Interim Evaluation, and as mentioned above, the
Operating Structure might appear slightly weak in know-how on implementation of Active Labour
Market Measures and design of education action underneath the umbrella of Demand-driven
Education and Training among other issues (including practical measures to ensure registration of
work places, implementation of Occupational Safety and Health standards, establishing occupational
and educational standards, labour market demand analyses and similar targeted studies, and
presumably even coordination of these initiatives). On the other hand, that does not indicate that the
practical know-how on these topics in not present in Turkey; the know-how is obviously anchored in
the Operation Beneficiaries.
32
The Interim Evaluation recognizes and acknowledges the design of the management structure of the
HRD OP which appears highly plausible from an organisational point of view, but still the Interim
Evaluation fear that conflicts in measures, actions, policies, and perhaps even targets might arise
without a solid, technical, cooperation between the main Operation Beneficiaries, hopefully boarded
and controlled by a strong chair to avoid the almost global and everywhere present conflict in means
notably between the two main policy areas of National Education and Public Employment Service 9.
The question becomes trickier when it comes to turn studies of demand into something workable for
the educational sector, where İŞKUR still lack basic tools and techniques while the educational
institutions suffer from relative high autonomy, notably as the VET/TVET sector is primarily founded on
university grounds. The universities of Turkey may conduct their own training needs analyses and
studies of local and regional demand, but the ability to see future demands as well as the ability to
induce knowledge sharing across regions (allowing mobility of skilled labour to become a player in the
fulfilment of needs) is hampered by autonomy and the subsequent lack of consensus in methods to
display the main features of the current and the future labour market.
Again, we have to recall that studies of labour market and labour market demand and shortages are
not the core expertise of the educational sector, and notably not the HE sector. In that respect a high
cooperation with İŞKUR is needed in order to ensure sustainability but not least in order to ensure a
sustainable economic development in the regions.
In that respect the communication between the two main players are yet to be proven for real (just like
in many other countries). And perhaps the TAT was able to spot some sort of mismatch between the
two organisations insight in the subject combined with their proven large experience with international
projects, and the weak Operating Structure, not least when it comes to establish a viable agenda on
other issues that formal and doll procedures and reporting routines.
There is no doubt that both organisations contain efficiency and effectiveness, notably when it comes
to content. They are both experienced in procedural issues. Somehow exactly these facts may be a
reason for the observed strange holding back towards the Operating Structure, allowing the Operating
Structure to set the agenda for the HRD OP.
The Social Security Institution, SSI, involvement is personalized in one single employee. It appears
fruitless to make SWOT on him if not directly inappropriate. The Interim Evaluation did not receive
data from SSI, as mentioned a couple of times; hence the evaluation of the data handling and
management of monitoring and evaluation cannot be examined.
9
The reference to global level, which includes EU Member States as well, is not just theoretic. Most countries
experience more or less permanently conflicts in vision, objectives, measures, targets, target groups, between the
two main Ministries. Some countries have tried to combat the conflict by merging the two ministries unfortunately
with limited success. The Interim Evaluation has accordingly no doable solutions or recommendations to offer
from Best Practice experience except from the fact that most EU Member States have experienced still growing
integration and interaction (perhaps through technical cooperative bodies) between the two Ministries; a process
which has shown to take long to obtain and a process which have never been without conflicts.
In Turkey an additional hurdle may arise from the fact that most of the professional VET and TVET educations
and courses are provided by strong and more or less autonomous Universities.
33
4.1.3 The Social Partners and the Chambers
SWOT assessment of Social Partners and Chambers
Strengths

Solid knowledge of project implementation

Strong commitment

Still stronger involvement and knowledge on
educational issues

Strong willingness to take part in the process

Flexibility and open, modern attitude towards
counterpart as well as the public sector
Weaknesses

The strong commitment is still mainly on
central level (Social Partners; the Chambers
work on regional level as well)

Especially the Trade Unions appear to have a
weak infrastructure and may not have an
exclusive focus HRD OP oriented regions and
localities

Basically no tripartite systems, bodies and
boards on regional level creates less active
involvement in all phases of the
implementation and monitoring of the HRD
OP
Opportunities

Involvement not just in HRD OP but also in
establishment of NQF will gradually increase
the demand for still more active involvement
of the Social Partners from a professional
perspective

Better dialogue between the main actors on
the labour market decrease the risk of
conflicts

Better dialogue between the main actors on
the labour market increase the potentials for
inclusion, including:
o Registered employment (decrease
the informal sector)
o Social inclusion of disadvantaged
persons

Better dialogue between the main actors on
the labour market increase the potentials for
inclusion of other main themes, not least
matters of OSH (which is completely left out
the HRD OP)
Threats

The main threat is the lack of will power to
build a formal structure for Dialogue and
participation on regional level (first) and local
level (next). In that respect the main actor will
be the public sector which has the obligation
to construct the fundament. In here, the
legislative fundament should be brought up.
The main players on the labour market and in all matters of HRD in Turkey as well as in the HRD OP
are mainly subject for examination in the Thematic Report on Horizontal issues to which the Interim
Evaluation Report therefore refer.
4.2 The Monitoring Arrangements and the Monitoring Information System with
respect to the Grant Monitoring
4.2.1 Central Grant Monitoring Teams
The Central Grant Monitoring Teams, CGMT, are founded within İŞKUR (11 permanent and full time
staff members of the unit), MoNE (approximately the same staff number, but not all are full time), and
SSI (one person full time). With respect to SWOT tables and the broader description of the
organisation we refer to the previous section above. The section mainly focuses on the Central Grant
Monitoring Teams with the three organisations which functions is broadly specified above.
We briefly mentioned the weaknesses of the Operating Structure in terms of knowledge of content
within HRD issues which is reflected in the expected taking the lead in the programming of the HRD
OP. Similarly, we commented the apparent weakness in the management structure due to the silent
parties of İŞKUR and MoNE and their somehow natural different approach to these issues (which just
34
as naturally brings some conflicts on-board). These weaknesses, which appear to be generally
acknowledged, should accordingly be met by a strong involvement on behalf of the CGMTs.
That part appears to be fulfilled with solid expertise, knowledge, and experience from all three teams.
Nevertheless, the TAT have been unable to see the influence of the three CGMTs in programming of
the HRD OP including not least defining the outcome and the content of the programme, even though
all three bodies appear to agree with the priorities as well as the execution of the HRD OP. The point
is that the TAT would have expected the CGMTs to have a far more active role as key sparring
partners for the execution of the projects and their main outcome.
Of course, that viewpoint is only accentuated by the study of the indicators, which indicators all three
CGMTs were soon to criticise. In that respect, the TAT has measured the influence of the CGMTs (as
well as the mother institutions) on the programming of the HRD OP by the study of the outcome and
the way the complete programme was anticipated to be evaluated. Since the lack of impact indicators
make every evaluation impossible the TAT can only conclude that the CGMTs have had very limited
impact on the overall design of the programme10.
In fact, it appears that the main role of the CGMTs is to act as a prolonged monitoring function
supervised (?) by the M&E Unit of the OS11. If that is true something is wrong and the sustainability is
deeply threatened; obviously the CGMTs should have far more influence on the selection of projects,
the content of the projects and their contextual route, as well as the way the outcome of the projects
should be assessed12. The TAT, however, have no documentation which should point in that direction,
and can only presume that the CGMTs (and their mother organisations) have determining impact on
the HRD OP and have the ultimate power to set the agenda for the HRD OP, including the agenda of
the SMC Meetings13.
The Interim Evaluation is not in any position to question the structure of the HRD OP management
system. Likewise, the Interim Evaluation has no reason to question the willingness to use the HRD OP
as a solid platform for structural changes in matters of HRD. The Interim Evaluation can only raise
comments based on reflexions and observations from the examination of the system as impartial
outsiders.
It appears that the overall management structure put a lot of effort in procedures and financial
management on the expense of ensuring discussions of content, outcome and impact and how to
address the regional needs. That viewpoint is further reflected, and perhaps even accentuated, by the
study on indicators as well as the assessment of the involvement on regional level.
Accordingly, the Interim Evaluation has no means to make firm conclusions; the logic from a content
part is missing or at least the TAT sees broken links which appear to contradict the well-proven and
ready-designed structure of the HRD OP. And the TAT has no means to prove and document on the
observations.
However, the Interim Evaluation strongly recommend the CGMTs to take on board more ownership in
terms of identifying the purpose and the means, measures and actions of the programme in order to
ensure sustainability and compliance with regional needs. In particular, the Interim Evaluation
10
Unless, of course, that the CGMTs lack knowledge of measurement techniques which indeed would appear
extremely strange due to their long term experience with initiatives and international projects.
11 Indeed that is the way the function of the CGMTs have been presented to the TAT
12 The Operation Structure has the following comment to the reflexion of the TAT: “This is not true. The ME Unit is
not carrying out monitoring functions at grant level. As there are many monitors at regional and central levels and
they make the visits together, the ME unit is just coordinating the preparation of the monitoring visit plans.”
13 Which certainly is not the case as far as the TAT has observed, which just raise the question of the main
reason for establishing the SMC if the SMC function is procedural exclusively (see the Thematic Report on
Horizontal Issues).
35
recommends the CGMTs to take a far more active role in determining indicators which reflects real
effect (outcome and impact), in ensuring solid means for monitoring and follow-up, and perhaps even
to ensure strong dialogue with stakeholders in each of the single projects under implementation.
As the Interim Evaluation sees the organisation, the CGMTs and their mother organisations have the
responsibility to ensure that the programme reflects real needs. The CGMTs should, according to the
Interim Evaluation’s view, have the necessary experience, knowledge and know-how to embark on a
far more prudent involvement in the programme and its design, perhaps not during the 2007-09
programme, but at least in future HRD OPs.
In that respect it will be of paramount importance that the OS within the MoLSS embark on a dialogue
with the CGMTs in finding ways to ensure a stronger commitment and ownership in the programme on
behalf of the CGMTs and their mother organisations. The same, by the way, goes for the involvement
of the Social Partners, which we noted above.
4.2.2 Regional Grant Monitoring and Technical Assistance Teams (RGMTTs)
The results of the examination of the RGMTTs are covered by the thematic report on Horizontal
Issues to which the Interim Evaluation Report hereby refers.
4.2.3 The Monitoring Information System
The Monitoring Information System, MIS for short, has been being developed and updated since
2006. The use and, notably in this context, the commitment of entering data and providing updates
and inputs to system is not compulsory. In order to ensure consistent input to the system monitors (the
RGMTs) have been assigned indicating that the feed in to the system still necessitate a back-up
organisation. Some difficulties appear even on the present structure, though, mainly due to the
limitations of the RGMTs members (they do not work full time among other things; see Thematic
Report II).
Accordingly, the system suffers from some shortcomings which include pitfalls in quantifying and
updating of the agreed indicators, in providing project-specific data and perhaps reliable information
on the current state of the projects. The system, still under development, accordingly suffers from
incomplete data.
In the examination of the MIS the Interim Evaluation has noted the complexity and feasibility of the
system, mainly when it comes to providing information and data on administration and financial
progress. The system is solidly used by the Operation Structure and appears to have become an
important tool for the overall management of the programme.
However, the Interim Evaluation has also discovered and experienced its shortcomings. In the study of
indicators for follow-up on the programme several difficulties appeared, and the overall conclusion
from the study on indicators is that data still has to be provided from the main central stakeholders,
mainly the CGMTs and their organisations.
There are several reasons for this:
36
1. Data on some indicators are simply not there and the data, which are there, are either
incomplete or not directed towards the indicators14
2. The reporting to the system do not appear to be conducted consistently when progress is
made and identified;
3. Feeding in to the system is not compulsory indicating that the prime caretakers have only
limited interest in supporting the system
4. Some, if not a large part, of the indicators are not workable in practice for monitoring and
evaluation purposes and has to be redefined in order to become quantifiable or in order to
ensure the correct measures and data (see the study on indicators)
5. Data and the quality of data appears to vary across operations, projects and regions;
6. Data lacks basic background statistics where background statistics are needed for
assessments of indicators; and it appears extremely difficult to see how the MIS data links to
other statistics, including official statistics and administrative records 15
7. The MIS lacks updated statistics and information from other sources relevant for the indicators
The Interim Evaluation recommends to







Upgrade the system to be of benefit for all stakeholders (project contractors sees limited
benefit from entering data on a daily basis)
Consider to make the system compulsory for contractors and other key stakeholders 16
Ensure that the system actually reflects the need (the issue of the indicators, which basically
has to be redefined)
Ensure interlinks with basic data, including official statistics and administrative records when
needed for measurement
Ensure solid training in quality assurance on all levels, notably micro level where data are
entered
Clean all records in close liaison with caretakers of administrative records, notably SSI,
İŞKUR, and MoNE
Make quality controls on a permanent basis
4.3 The use and financial allocation of financial assistance and the financial
management of the budget
Overall comments on financial management
The overall financial management system is functioning well. The flow of funds from the EC level till
the final beneficiary level is smooth. While this is not specifically subject to this evaluation, the financial
aspects are covered well with relevant legal documents and reporting systems.
Commitment and disbursement data
Commitment rate shows the amount of funds contracted compared to total available budget of a
programme and thus provides an indirect indication of the progress of a programme. It is a more
14
Agreed, one of the main problems is that the indicators are defined wrong. Most of them includes words like
increase and especially when it comes to Result indicators the indicators includes rates without specifying the
denominator (and without having the ability to capture the most plausible denominators within the system).
15 At least it has shown to be extremely time-consuming to find links between project data and administrative
records; the study thereof ended up inconclusive, though the TAT tends to believe that there is no link
whatsoever.
16 This point should go in line with a dialogue with the stakeholders on how to make the system both user-friendly
(if that is a real issue for the stakeholders) and informative (e.g. cross-regional or cross-project information)
37
valuable indicator for programmes in case of which the deadline for the conclusion of contracts
precedes the deadline for corresponding payments.
The commitment rate of 60.33% at less than two years from the beginning of the commitment period
i.e. the signature of Financing Agreement should be normally considered satisfactory in the context of
the n+3 rule.
Disbursement rate shows the amount of funds paid to contractors compared to total available budget
of a programme giving indirect information of the overall physical progress at a certain point of time.
However, as according to the contractual rules payments are not always linked to physical outputs due
large advance payments and delayed final payments, the disbursement rate does not allow to
measure the physical progress of a programme very accurately.
Disbursement rates could also give an indication of a possible risk of decommitment, but under the
n+3 rules with complex decommitment deadlines the disbursement rate proves less informative.
Similarly with the commitment rate, the disbursement rate of 40.32% at less than two years from the
beginning of the commitment period i.e. the signature of Financing Agreement, should be normally
considered satisfactory in the context of the n+3 rule.
National co-financing and EU contribution
The national co-financing and the EU financing have been available in a timely manner. Payments are
made in accordance with 85/15 ratio between EU and national financing. This also indicated by
identical commitment and especially disbursement rates of both EU contribution and national cofinancing.
Payments to contractors
While the evaluation team did not have access to specific financial documents, interviews with some
contractors indicated that payments to contractors have been made in a timely manner, although
mostly closer to the end of the legally allowed deadline for payments. The contracts provide legal
sanctions for delayed payments which in turn should be a sufficient guarantee that the payments are
made in a timely manner.
Shorter implementation periods and decommitment risk
According to the Financial Regulation and the Financing Agreements each programme should be
implemented within a specific period to be eligible for EC financing. In the event that the allocated
funds are not used within a prescribed time period, the EC decommits the funds, i.e. the funds can no
longer be used for the intended purpose even if the beneficiary country has signed binding contracts
to be financed for these funds. The HRD OP is also subject to that “n+3 rule”, which in very broad
terms should enable to have an implementation period of more than three years for the
implementation of a programme. For multi-annual programmes like the HRD OP the implementation
period is even longer although having several interim deadlines.
In case of the HRD OP 2007-2009, the implementation period lasts for three further years from each
of the three programming years. Due to different rules applied e.g. to the pre-financed amounts
received from the EU and the end of the implementing period, the interim deadlines are slightly
different in reality. In broad terms, the minimum amounts of EC funds that have to be spent in
accordance with specific requirements are the following:
38
Year
Budget (MEUR)
End of 2010
2.59
End of 2011
52.9
End of 2012
55.6
End of 2018
47.6117
These deadlines for the HRD OP 2007-2009 appear to leave ample time for the implementation of the
programme, however, it should be considered that it was possible to first use the funds under the HRD
OP only after the signature of the Financing Agreement on September 16, 2009.
Since accreditation process may become one of the contributing factors to the possible decommitment
of funds, the evaluation team analysed also the chain of events that caused later signature of the
Financing Agreement.
According to the IPA regulations, the funds under IPA Human Resources Development component
are accessible only to candidate countries accredited to manage funds in a decentralised manner.
Therefore, the candidate countries, including Turkey, had to undergo the process of preparations for
the accreditation, a process expected to take up to three years based on the experience of other
countries.
Table. Key steps in IPA accreditation process in Turkey
Time
Activity/Event
July 2006
Publication of the IPA Framework Regulation (Council Regulation (EC) No
1085/2006 of 17 July 2006 establishing an Instrument for Pre-Accession
Assistance (IPA))
March-April 2007
Gap Assessment study performed
June 2007
Publication of the IPA Implementing Regulation that stipulated the criteria
required for the accreditation process (Commission Regulation (EC) No
718/2007 of 12 June 2007 implementing Council Regulation (EC) No
1085/2006 establishing an instrument for pre-accession assistance (IPA))
August-October 2007
Gap Plugging
December 2007 – July
2008
Compliance Audit
September 2008
National Accreditation
October 2008
Accreditation packages sent to EC
November-December
2008
Verification Audits by EC
January 2009
Verification Audit Report issued by EC
March-June 2009
Further Verification Audits by EC
August 2009
Conferral of management powers by EC
September 2009
Signature of Financing Agreement
The table indicates that the decision on Conferral of management powers was made approximately
two years after the publication of the accreditation criteria that established the basis for the
requirements to be fulfilled. Based on the experience from similar accreditations in other countries
17
Maximum amount not taking into account any previous decommitments under the programme
39
before, an approximately two-year period for the completion of the accreditation process could have
been foreseeable at the time of publishing the accreditation criteria.
At the same time, the first OPs in the candidate countries started with the budget year of 2007. While
the complexities and expected duration of the accreditation were foreseeable, no grace period or
transitional mechanism was provided. As a result, for the first years of implementation, the principle of
n+3 was in reality transformed into n+2 or n+1. This may not be the (single) reason leading to possible
decommitments in candidate countries, but obviously the implementation periods for the first years of
HRD OP are shorter.
Consequently, any decommitments in the first years of HRD implementation should not lead
automatically to long-term conclusions about implementation capacities in the candidate countries as
in future years with longer implementation periods decommitments can be more easily avoided.
Similarly, the shorter implementation period should not be automatically seen as a single reason for
any decommitments in the candidate countries, but other aspects should be equally analysed in this
context.
In case of HRD OP in Turkey, a relatively minor amount of 198 591 EUR was decommited at the end
of 2010. However, in 2011 the amount to be declared to the EC is 52.9 MEUR and according risk
assessments, at least some of the amount would probably be decommitted. However, according to the
same risk assessments, there will be no decommitment risk for any part of the amount of 55.6 MEUR
at the end of 2012. This may indicate that all other conditions remaining the same there would have
been no decommitments if HRD OP could have been implemented following the n+3 rule without time
limitations arising from the accreditation process. Despite the limitations coming from the accreditation
process during the first years of the HRD OP, the Turkish administration and the EUD have taken
measures to minimise the risk of decommitments, by e.g., launching calls for proposals and tender
with a suspension clause prior to the signature of the Financing Agreement. These activities clearly
indicate the good will, cooperation and foresight by the institutions involved.
At the same time, this does not necessarily mean that there is no room for improvement in the
processes. These issues are addressed in other parts of this report.
4.4 Review programme outputs and results in relation with the expenditures
incurred at priority and measure levels; in the light of financial and physical
progress
The task of reviewing programme outputs and results in relation with the expenditures incurred at
priority and measure levels; in the light of financial and physical progress could be best performed
after the projects under a measure or priority have been completed and the relevant statistical data is
available. By the cut-off date of April 30, 2011 of this evaluation project, none of the projects were
completed making this assessment difficult.
The financing model of 80% of advance payment and 20% of final payment that is not linked to
physical progress makes it impossible to make such an evaluation before a project is completed.
Therefore, this task should be addressed by the following evaluations. However, although the cut-off
date for this evaluation has been set for April 30, 2011, the project can still accommodate the partial
analysis of the grant scheme for the Promotion of Life Long Learning in this report that was completed
in May 2011. Consequently, the data from the beneficiaries was still continuously updated in the MIS
in the course of preparing final reports and closing the administrative side of the project after the end
of the implementation period. The monitoring data is, thus, incomplete and therefore the analysis can
not be completely valid, however a preliminary analysis is included in the Annex.
40
However, the TAT could make some general observations in the course of desk study and interviews:
-
-
The market of external trainers used in grants projects appears to be quite volatile. In some
cases the fees of trainers have exceeded anything that could be considered feasible even by
international standards. While this area could be difficult to regulate, it would be advisable to
see that any such fees would be not be unnecessarily excessive.
Since the national vocational education standards are under preparation, it is not ensured at
this stage that the funds spent on vocational training would result in a training that would be of
sufficient quality. Equally the certificate provided at the end of the trainings might prove thus
worthless for the participants.
4.5 Examine the efficiency of Operation Selection Criteria
According to article 158 of the IPA Implementing Regulation, all operations which are not major
projects and which are implemented by final beneficiaries other than national public bodies shall be
selected through calls for proposals. The selection criteria shall be drawn up by the operating structure
and shall be published with the call for proposals. In other words, the selection criteria will be used in
grant schemes and published with the Call for Proposals.
There have been five grant schemes launched under the HRD OP under five different measures:
Measure
1.1.
1.2.
1.3.
2.1.
3.1.
Grant Scheme
Promoting Women’s Employment (PWE)
Promoting Youth Employment (PYE)
Promoting Registered Employment through Innovative Measures (PRE)
Increasing School Enrolment Rates Especially for Girls (IER)
Promotion of Life Long Learning (LLL)
The selection criteria have been included in the Calls for Proposals and also at a more general level in
the HRD OP. The term “selection criteria” may be used in a wider and more specific sense, but in the
context of this evaluation is should include also the criteria published as “eligible actions”, which do set
the scope of projects financed under the grant schemes.
Since the selection criteria are mostly similar, the for all measures (obviously with the exception of the
main area), the criteria for measure 1.1. have been used as a sample. The criteria in the HRD OP and
Calls for Proposals are not completely identical, but fall under the objectives of Measure 1.1. In both
cases, the set of criteria is rather broad and open and allows the applicants to propose rather different
project ideas. Such a policy may be justified considering that this is the only grant scheme in that area
at a given point of time. The method of using rather broad priority themes with a rather long nonbinding sample list of activities should be considered justified in such a situation.
The disadvantage of a system is that the selected set of grants projects could become very diverse in
the end focussing on different sub-themes in different areas. This approach could thus dissolve the
results and outputs in such a way that it would be difficult in the end to measure common impact.
In the future under Structural Funds with larger amount of funds that could have measurable socioeconomic impact, it might be advisable to use more targeted grant schemes. However, as the HRD
OP is in a way a pilot scheme and a preparation for the management of structural policies with limited
funds, a broader set of themes may be justified. However, it should be then considered that it would
make final evaluation of the programme more complicated.
Equally, while this would be outside the scope of the current interim evaluation project, studying the all
the applications submitted under the call for proposals could reveal how diverse the project proposals
41
were in content and to what extent it complicated the comparison and selection of project proposals.
This may be a subject for another study in order to find out if such a broad definition of eligible actions
could create to a certain extent a “lottery effect” in the project selection process.
The evaluation team has similar comments to the selection criteria of other calls for proposals and
therefore these will not be analysed here. This analysis based on the call for proposal under measure
1.1. to equally valid for other calls for proposals under other measures.
The budget brackets for the grants can in general considered optimal. However, with the current
budgets the number of grants is relatively high, which makes monitoring of the grants more complex.
At the same time, due to the requirement of own contribution by beneficiaries, increasing the minimum
budget requirements may close force some beneficiaries out of the market, as the amount of own
contribution is considered to be high already now even by some larger and more capable institutions.
4.6 Assessment of local absorption and implementation capacities
HRD OP has a strong regional dimension and therefore assessment of the local absorption and
implementation capacities could give valuable inputs for future programmes and implementation
practices. It must be still said that since the current assignment is carried out as an interim evaluation
at a relatively early stage of the implementation of the programme, the assessment can be conducted
on the basis of a rather limited sample. However, certain practices and commonalities can be
observed.
Local absorption and implementation capacities can be assessed on the basis of five regional grant
schemes being implemented in the regions at the start of the Interim Evaluation.
The grant schemes are the following:
Measure
1.1.
1.2.
1.3.
2.1.
3.1.
Grant Scheme
Promoting Women’s Employment (PWE)
Promoting Youth Employment (PYE)
Promoting Registered Employment through Innovative Measures (PRE)
Increasing School Enrolment Rates Especially for Girls (IER)
Promotion of Life Long Learning (LLL)
Assessment of local absorption capacities
Based on the experience of the five grant schemes, different aspects of absorption capacity can be
assessed by using the following data.
-
The number of applications received from a specific region indicates the activity level and
awareness of the HRD OP grant schemes by the applicants.
The percentage of applications funded of the total number of applications submitted from one
region indicates the average quality of applications from a specific region.
The average number of projects selected per one million inhabitants from each region
indicates how the funds have been distributed per population unit between the regions.
The statistical data annexed to this report provides a good basis for such analysis. All the data should
be still analysed broadly by looking at tendencies rather than at specific figures as there are several
external aspects that influence the figures. First of all, although the regions are of comparable size in
terms of population, they are not of equal size. In the light of that, the data can be analysed as e.g. the
number of projects financed per e.g. million inhabitants in a specific region. While this adds an
42
additional dimension, the limited number of projects per regions and a certain extent of subjectivity
that is always involved in grant evaluations would add in too many variables for too specific and
detailed conclusions.
Since regional absorption capacities should be analysed in the context of the whole HRD OP,
individual grant scheme data will not be focussed on in this part of the report. Instead, the data is used
in the evaluation in combination with other information, including observations made by the evaluation
team during the visits to different regions.
It must be also emphasised that the evaluation team has not had access to individual grants selection
documents nor has it visited all the regions. While this also could not have been within the scope of
the HRD OP evaluation, the findings and conclusions in this part of the report are based on available
statistical data and observations based on samples. For a detailed analysis taking into account the
specificities in each region, a separate study could be considered.
Overall, the data indicates rather normal distribution of grants per region and per population unit.
Without a regional quota system there are natural differences in the number of projects per region or
per an amount of population. Still, in some cases the variations are still notable. For example, the
NUTS II regions of Kars, Batman and Gaziantep receive a lower amount of projects per population
unit than other regions.
Also, the percentage of the submitted applications that have been selected for financing appears to be
lower in the regions that receive less funding. In other words, while the number of applications is not
low, only a relatively small percentage gets selected. This indicates that the awareness of the grant
schemes is still at a reasonable level but the quality of applications is lower than the quality of
applications in more successful regions.
Another remarkable case appears to be the NUTS II region of Van that has a relatively low percentage
of applications that have been selected for financing. However, in that region the number of projects
per population unit is one of the highest and this is also reflected by the relatively higher number of
applications submitted. This indicates that the awareness and activity level is very high in the region,
but the average quality of applications as compared to other regions is lower.
Grant schemes are competitive in nature and without a regional quota system lower rates of selected
projects need not necessarily indicate a much lower quality of applications. It may equally indicate that
simply projects from some other regions have been of slightly better quality.
It also appeared from the regional visits that the regions with a strong proactive central institution as
well as with strong project writing capacity tend to do better in the project selection process.
In some regions, there are institutions (e.g. Chambers of Commerce, Regional Grant Monitoring and
Technical Assistance Teams) that are taking strong proactive measures to promote the grant
schemes. Awareness raising events, targeted contacts with institutions that could be successful
applicants but lack specific procedural knowledge, advice on project drafting and implementation are
some of the practices that have contributed to higher regional success rates.
Another success factor appears to be strong project drafting capacity in a region. In some regions
universities or other institutions have trained master students or other interested people on project
drafting skills. Regions, where these activities were more prevalent tend to have better success rates
also in the statistics.
43
Therefore, one of the reasons for higher concentration of projects in some regions could be linked to
the fact of “global competition” between the projects. As a simplified example, if two regions are
competing for 10 projects and one region has all the applications only of a marginally better quality
then that region could still win all the projects while the other would get none. If such tendencies
appear in regional socio-economic programmes in case of which reasonably equal regional
distribution of funds should be one of the aims, some moderate regional quota system not having a
significant effect on the quality of projects, could be considered. For example, a solution by which the
half of the funds will be distributed based on regional quotas, provided that the project meets certain
quality standards and the rest of the funds could be distributed based on “global competition”.
A further factor that could influence local absorption capacity is related to the possibilities of potential
applicants at the time the calls for proposals are launched. For example, according to grant
beneficiaries, they have not applied for grants for time reasons if several grant schemes have been
open in parallel. Equally, some have skipped application if they have not had funds for co-financing
because of cash-flow issues, e.g. when waiting for the approval of the final report and subsequent
payment for a previous project. This factor should not cause however differences between regions, but
could lower the absorption capacity in general to a certain extent.
Assessment of local implementation capacities
Local implementation capacities can be measured mostly on the basis of the implementation
experience of the five grant schemes in the 12 NUTS II regions. Since at the start of the evaluation,
none of the grant schemes had been completed, the data for making fundamental conclusions is not
sufficient. Still, some observations could be made.
Overall, the implementation capacities in the regions depend on the experience of specific grant
beneficiaries. The grant beneficiaries have a reasonable external support structure that they can also
rely on. The grant beneficiaries may contact RGMTTs, CGMTs, the relevant ministries and the CFCU.
Equally, they can use the Grant Monitoring Information System for support. In some regions, some
local organisations provided further advice.
While the grant beneficiaries should address first the RGMTTs this practice is not always observed. In
such cases, other counterparts often do not receive the information and remain unaware of the issues.
Local implementation capacities can be more issues the grant beneficiaries face are of procedural
nature and therefore they have developed a practice of contacting directly the thoroughly assessed
after a number of the grants schemes have been completed and the respective final reports approved.
This may be subject to a further study in the future.
4.7 Analysis of the accuracy of the budget allocations under measures
The wider aim of the HRD OP is both to prepare Turkey for the implementation of ESF type of
measures functioning as a pilot scheme and within that frame address more specifically the areas of
human resources development that would address the most acute needs of the country. The Annex
provides an overview of the division of funds between priorities and measures as well as regional
coverage.
Considering the aims of preparing for the ESF implementation and addressing the key priorities in the
human resources development sector, such a division of funds is justified. The areas of employment,
education and social inclusion are all covered for the purposes of HRD OP serving as a pilot scheme
44
with a relatively larger amount dedicated to the more prioritised employment sector. The budget
allocation under measures corresponds to the breakdown of the budget between the priority areas.
The territorial breakdown had been also reasonably planned. With a clear need of assistance at this
pilot phase on the central level, the majority of funds are still targeted to the NUTS II regions. The
breakdown between the NUTS II growth centres and hinterlands reflects broadly the division of
population between the two in the regions. For the purposes of a pilot scheme such a division is
justified, however, for future programmes a different balance could be considered, if targeting specific
population groups would bring better results.
The division between the growth centres and hinterlands has been planned as a range rather than
fixed percentages. Provided that the ranges are not too broad, which they are not in case of this HRD
OP, this budgeting method should be encouraged for practical reasons. Namely, the grants schemes
under HRD OP are addressed jointly to the growth centres and hinterlands and in the project selection
process a certain level of flexibility on regional level is beneficial in order to finance the most relevant
projects. Fixed percentage could unnecessarily complicate the evaluation process.
The experience with five grant scheme launched under the HRD OP shows that using no quotas
between the regions could lead to relatively uneven distribution of projects between the NUTS II
regions. Therefore, it would be worth considering introducing setting minimum quotas for each of
future grants schemes while allowing still sufficient global competition and maintaining minimum
quality levels of the projects. It would not be advisable to use fixed divisions between the NUTS II
regions on the level of the HRD OP, as this could unnecessarily complicate planning and
implementation of some projects.
4.8 The study on Indicators
The Interim Evaluation has been able to recognize some 95 Output indicators from the HRD. In
addition we further investigated 28 Result indicators, which can be divided into the following:
•
9 indicators are Output indicators
•
The remaining 19 indicators are all Outcome Indicators of which:
o 10 Outcome Indicators are workable in practice
o 4 Outcome Indicators could work with some light adjustments
o 5 Outcome Indicators cannot work in its present form and needs to be adjusted
o We found not even one single Impact Indicator
In total we have identified 104 Output Indicators, 19 Outcome Indicators and 0 (zero) Impact Indictors.
Of the 19 Outcome Indicators, ten of them passed the Evaluation without remarks, 9 of the indicators
can be workable with some adjustments. 5 Result Indicators cannot work in practice.
Obviously, we are talking about a very high number of indicators, which makes it close to impossible
to undertake evaluation reports; each indicator will have to examined in time series analysis
throughout the implementation and the following years, and each of them has to be examined in
various cross-sections studies, not just on regional levels but toward target groups and control groups.
Using a conservative estimate based upon experience we could easily suggest that each indicator
would require close to 5 tables, with presumably another 5 tables in annex, and close to 6 pages
description. Due to the number of indicators it would generate evaluation reports of 700 pages, which
obviously would take months to produce. And yet the HRD OP does not even have any Impact
Indicators making it impossible to evaluate the effect of the HRD OP.
45
Keeping track of a record of 104 output indicators is very appreciable, but no one bother read the
lengthy report the evaluation will create in that case, basically indicating that the system creates a
sequence of reports where only a fraction of the conclusions will be read, understood and acted upon.
In that respect we have to recall that the burden of monitoring and evaluation does not fall on single
M&E experts on central level; the burden is transferred to the complete system, perhaps in some
cases even to final beneficiaries; I.e. finding and creating data, controlling data, ensuring timely input,
ensuring quality procedures in order to produce data of sufficient quality, follow-up, and making
comments.
The programme lacks Impact Indicators. Without impact indicators it becomes impossible to see the
real effects of the programme. By real effects we mean the impact on the local / regional economy, the
labour market and the welfare.
Impact Indicators measures the consequences of the outcomes in terms of wider objectives
(for example literacy rates, health improvement). The definition covers the wider effects of the
outcomes but there might also be higher level impacts, related to broader objectives – growth
and income poverty, for example. In order to turn it into a net impact indicator the effect of
other economic, educational, and employment programmes have to be assessed and
accordingly subtracted.
The Impact indicators will become vital for the final evaluation of the programme. The final evaluation
will require means and tools to do a fair and solid evaluation of the impact of implementing so far 431
projects in 12 NUTS 2 Regions. In order to make the impact assessment data should be secured for a
long time period ensuring solid means to do impact studies across time. And the indicators should be
agreed upon long before the evaluators arrive; it would be impossible for the evaluators to establish
the evaluation criteria and the indicators on top of the impact assessment.
The complete lack of impact indicators tells us that the difference between output, outcome and
impact has never been understood by the stakeholders.
Conclusions and recommendations on the study of Indicators
In the Sectoral Annual Report on the Implementation of the Human Resources Development
Operational Programme, 2010, the following comments to the functionality of the indicators were
found:
“…some difficulties were encountered in determining the actual situation in terms of
achievement of indicators. Although there were some problems stemming mainly from the fact
that practice of monitoring the indicators was at its initial stage, these kinds of preliminary
studies were beneficial from many aspects; for example, importance of entering the
“performance indicators” were emphasized by the MEU and the RGMTT Experts, and the
grant beneficiaries were requested to enter the targets and status of achievement in terms of
indicators determined within their project. In addition, some revisions were made in the
indicators in terms of expression and wording, also some indicators were divided in different
parts for converting the indicator into a measurable format.”
[SAR, p. 38]
Somehow, the Interim Evaluation finds the adjustments insufficient. At least the first and perhaps
prime recommendation is…:
… to revise the current set of indicators, which revision should bring the number of indicators
significantly down. The need for a reform is urgent since the current system of indicators are
46
insufficient, inappropriate and yet far too ambitious. The Interim Evaluation proposes to bring
the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact
Indicators18.
I.e. the current system is:
•
Insufficient since the current set of indicators lack Impact measures
•
Inappropriate since many indicators have no doable baselines, lack data, lack
infrastructure, or are just not measuring the right (defined wrongly)
•
Too ambitious due to the number of indicators
The Interim Evaluation has of course suggestions to how ease the administrative burden of a far too
ambitious monitoring and evaluation programme. The suggestions fall in three main categories:
1.
2.
3.
Revise the indictors toughly until a decent and doable target is reached 19; Understand the
basics in evaluation and design a list of indicators which summarizes the main and basic
idea and purpose of the overall programme;
Establish a multi-level system of indicators, ensuring that the main evaluation reports only
contains a very small number of smart indicators. The multi-level system does not
decrease the overall workload of the complete system, but at least the main work falls on
the central level;
Establish solid training programmes in monitoring and evaluation, data control, and quality
assurance, not least on regional level. Place ownership and delegate task where the
single stakeholders can see the purpose.
The first suggestion is surely the hardest one. On the other hand the TAT has been informed that
many central level key stakeholders have never seen the present set of indicators 20. And besides, it is
not just a matter of a far too long list of indicators; the complete programme has no means to make
evaluations due to the complete lack of impact indicators.
The second solution, which draws upon the introduction in Chapter 7, is further explained in a section
below.
The third suggestion leads to open up for a far more flexible administration where it perhaps would be
fair to delegate monitoring and even parts of the evaluation to other stakeholders than the central level
Operation Structure. Agreed, it is exactly what the stakeholders try to achieve, but many of the
stakeholders cannot find the logic yet notably on regional level. And exactly that fact makes the
“burden” a real burden and difficult to create flexible solutions.
The three suggestions can easily be implemented simultaneously.
Two detailed tables of conclusions and recommendations are annexed to the study of Indicators in
Annex A4. The two lengthy tables (due to the number of Indicators in the present programme) could
be consulted in case the HRD OP intends to keep the present far too ambitious number of indicators.
Otherwise, the main recommendations from the Interim Evaluation are:

Revise the current set of indicators, which revision should bring the number of indicators
significantly down. The need for a reform is urgent since the current system of indicators are
insufficient, inappropriate and yet far too ambitious
18
The mix may vary slightly, of course
The study of indicators calls for a complete reform of the indicators. Perhaps it would be better to start from
scratch.
20 That goes, strangely, notably for the CGMTs by the way, which might be another point to add to our study of
the management structure. It is a complete puzzle why the central and key stakeholders were caught by surprise
when presented to the Result Indicators of the HRD OP. The Interim Evaluation did not follow up on the point
except from this very light note.
19
47




Revise the indictors toughly until a decent and doable target is reached; The Interim
Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output,
six Outcome, and six Impact Indicators
Understand the basics in evaluation and design a list of indicators which summarizes the main
and basic idea and purpose of the overall programme;
Establish a multi-level system of indicators, ensuring that the main evaluation reports only
contains a very small number of smart indicators. The multi-level system does not decrease
the overall workload of the complete system, but at least the main work falls on the central
level
Establish solid training programmes in monitoring and evaluation, data control, and quality
assurance, not least on regional level. Place ownership and delegate tasks where the single
stakeholders can see the purpose.
4.9 Overall assessment of the HRD OP
In general, the Interim Evaluation acknowledges the structure of the HRD OP which appear to work
and in good progress. In particular, the Interim Evaluation acknowledges:
 The first time with multiple projects
 The very new institutional and administrative setup
 The difficulties in getting the setup to manage the programme (it is Turkey; the largest country
doing HRD OP)
 The delays in the implementation of the HRD OP
 The rigid EU system and the not lesser EU procedures which will impose an increase in
bureaucracy in Turkey and will create challenges due to the very different administrative
culture in Turkey
 The wait for the accreditation of the OS
 The will power, the enthusiasm and the overall professionalism we have met
None of these points have been used in the assessments conducted, since that would not have been
fair.
The overall assessment of the HRD OP of Turkey is subsequently that the operations within the
programme are running and appear to be both relevant and in line with the intensions of the
programme as well as the national agenda and the national priorities. The management structure to
follow up appears geared and capable to handle a large scale programme, even though various
delays in the initial phase of the programme and perhaps not least the long wait for accreditation has
created significant challenges for the management.
In other words, the HRD OP has started and it works; the Interim Evaluation has found no
“disqualifying” features. Having said that, adjustments on a more detailed level appears necessary in
order to make a large scale HRD programme work efficiently in practice in the future and not least in
order not to over-burden the administration and henceforth create inflexible bureaucratic structures
which will only make the practical implementation of the single projects far more difficult. Professional
delegation to mainly regional stakeholders is the main weakness of the present system and another
main barrier for the administrative procedures is the almost intolerable number of indicators, which will
necessitate an overwhelming administrative burden both in terms of international follow-up and in
terms of the overall follow-up through monitoring and evaluation reports on behalf of the complete
programme and its operations. Some, if not directly many, of the indicators cannot be used in practice
and are not consolidated with neither the national statistical production nor the current system of
administrative records.
48
Accordingly, and in short, some elements or functions within are weakening the system to a critical
extent which appears to need attention notably in the longer run:
 Partnership approach, mainly on regional level
 RGMT setup, function and “ambassadorship”
 The MIS
 Too many indicators makes reporting impractical, if not directly impossible
 No impact indicators defined makes it impossible to measure effect
4.10 Recommendations
The main and core recommendations are as listed in the executive summary above and repeated
here. In addition, recommendations connected to the study on indicators follows from the study.
Concerning the efficiency and effectiveness of the HRD OP management structure:
1. Ensure far more pragmatism and flexibility based upon know-how and knowledge of “content”
instead of narrowly focusing on administrative procedures, which includes ensuring a far
stronger Social Dialogue, commitment, participation and involvement by all Social Partners on
all four levels of execution
2. Ensure stronger commitment and involvement from the Operation Beneficiaries.
Concerning the Monitoring Arrangements, notably the Monitoring Information System, MIS:
3. Ensure that the MIS system becomes more user-friendly or at least open up for a dialogue on
how to ensure consistent entering of data to the system
4. Ensure that the system actually reflects the need (the issue of the number and quality of
indicators, which definitely needs to be redefined)
5. Ensure a interlink with basic data, including official statistics and administrative records when
needed for measurement
6. Ensure solid training in quality assurance on all levels, notably micro level where data are
entered
7. Clean all records in close liaison with caretakers of administrative records, notably SSI,
İŞKUR, and MoNE
8. Make quality controls on a permanent basis.
Concerning the efficiency and effectiveness of the HRD OP contractual, tendering, and financial
procedures:
9. Ensure far more readiness and speed in the evaluation procedures and try to avoid conflicts in
central level responses to implementation level
10. Consider to establish quotas in order to ensure that all regions have at least a limited number
of projects reflecting the needs on the local level.
Concerning the efficiency and effectiveness of the HRD OP Indicators:
11. Revise the current set of indicators, which revision should bring the number of indicators
significantly down. The need for a reform is urgent since the current system of indicators are
insufficient, inappropriate and yet far too ambitious
12. Revise the indictors toughly until a decent and doable target is reached. The Interim
Evaluation proposes to bring the total number of indicators down to 18 indicators: six Output,
six Outcome, and six Impact Indicators
13. Understand the basics in evaluation and design a list of indicators which summarizes the main
and basic idea and purpose of the overall programme;
49
14. Establish a multi-level system of indicators, ensuring that the main evaluation reports only
contains a very small number of smart indicators. The multi-level system does not decrease
the overall workload of the complete system, but at least the main work falls on the central
level
15. Establish solid training programmes in monitoring and evaluation, data control, and quality
assurance, not least on regional level. Place ownership and delegate task where the single
stakeholders can see the purpose.
In case the HRD OP intend to stand up to the words of integration of the horizontal issues of
sustainable development and environmental protection the Interim Evaluation recommends
16. to integrate impact indicators in the programme for the ex-post evaluation to ensure that the
HRD OP has achieved to ensure sustainable development or least made progress in that
direction
17. to redefine and establish outcome indicators and impact indicators which particularly focus on
environmental protection
18. to ensure that environmental protection is acknowledged as a future demand in the
educational sector, notably on VET/TVET level (that could be part of new curriculum, for
instance)
19. to ensure that topics related environmental protection is an integral part of all Lifelong
Learning Trainings underneath Priority 3
20. to give priority to environmental protection topics in priorities and measures addressing gender
equality as well as social inclusion
21. to establish Training Needs Analysis (anywhere in the complete programme) which addresses
the topic of environmental protection both in order to grab a demand in time and partly to
increase awareness of the subject among the employers.
The Interim Evaluation has performed a solid investigation on participation. The study reveals
problems with participation on regional level mainly though some problems with the SMC construction
are evident. The long study of the participative approach ends with the following recommendations:
22. Support and continue the strengthening of the partnership approach on central level, not least
with respect to the structure of the SMC and its meeting procedures
23. Actively strengthen, perhaps through legislation, a far more viable partnership approach on
regional level.
And the regional level is subsequently examined where the main recommendations of the Interim
Evaluation are:
24. To ensure a solid platform on the regional level with a consolidated and consistent structure,
the build of a professional secretariat and governed by a tripartite body perhaps inspired by
the way the RDAs are constituted
25. Ensure “promoters” (ambassadors) are based regionally and that the promoters are able to
act as support in every phase. The TAT cannot see why the RGMTTs and the promoters are
not directly interlinked or perhaps even the same persons
26. Ensure means of commitment from Social Partners and other main regional actors on a
regional basis by placing ownership through direct involvement in the programming,
monitoring and evaluation phases
27. Ensure far more active involvement from both İŞKUR and RDAs in regional programming,
planning, design of indicators and targets;
28. Design objectives, measures, indicators and targets on regional level and establish a system
of reporting in accordance to a template designed from central level (which could feed directly
50
in to evaluation reports, like the SAR) to promote and reinforce regional involvement in the
HRD OP
29. Ensure that local members are part of the process (and not just representative from the
Growth centres) to ensure a far more active spread of the programmes internally in the
regions
30. Ensure gender equality, local representation, and representation of target groups in all
advisory boards to the HRD OP.
5. Annexes: Overview
The sequence of annexes to the Interim Evaluation Report falls in two main brackets or here Parts:
Part A consisting of underlying studies and Part B of background information connected to the
conducted evaluation. In addition, two Thematic Reports are annexed to the Interim Evaluation
Report. The idea of separating the background studies in Part A from the main report should be fairly
evident; the main point is that the background studies do not directly feed in to the main conclusions
and recommendations or are otherwise too detailed for main conclusion, albeit they all constitute an
integral part of the main findings. The humble hope is to increase the readability of the report.
PART A: Studies:
 Annex A1 Socio-Economic Study
 Annex A2 Major Developments on National and EU level since 2007
 Annex A3 Results of the ex-ante evaluation
 Annex A4 Annex to the study of Efficiency and Effectiveness
 Annex A5 The study on Indicators
PART B: Background Information
 Annex B1 Meetings held
 Annex B2 Terms of Reference
 Annex B3 Generic Question Frame for meetings
 Annex B4 Basic questionnaire on the functionality of the HRD OP
 Annex B5 Questionnaire on Indicators
 Annex B6 Survey Report on Basic questionnaire on the functionality of the HRD OP (in
Turkish)
 Annex B7 Survey Report on Indicators (in Turkish)
Thematic Report I on Complementarity between RC OP and HRD OP
Thematic Report II on Horizontal Issues
51
Annex A1. Socio-Economic Study
1. Introduction
The main purpose of the Socio-Economic Study is to provide references to the current situation in
Turkey and not least in its NUTS 2 regions. In that respect, the study is brief and kept to use a
minimum of indicators. Nevertheless, the study sets a scope for future evaluation of Impact of the HRD
OP. Hence the study serves as reference for the study of Indicators for the HRD OP in later chapters
of the Interim Evaluation Report.
The inclusion of data from EUROSTAT serves at least two purposes:
1. To provide a reference and ensure transparent statistics
2. To give reasons for the HRD OP mainly in terms of providing the ability to make Benchmark
studies
In accordance to best practice in studies on Human Resource Development the Socio-Economic
Analysis focuses on four main areas:
1. Demographic Indicators
2. Main Economic Indicators
3. Labour Market Indicators
4. Indicators on Educational enrolment and attainment
The Socio-Economic background analysis
TÜİK/TURKSTAT and EUROSTAT.
is
based upon official statistics
derived from
Not all statistics are available on the NUTS 2 Regions of Turkey. In particular, this study, which further
had the purpose of investigating potential impact indicators, could in particular have gained from
statistics on the following indicators on regional (NUTS 2) level:
1. Number of persons above the age of 23 (or perhaps 24) with at least ISCED level 3 by gender
2. Educational attainment levels (% of the population aged 25 and above having attained: a. less
than upper secondary; b. upper secondary or postsecondary non-tertiary; and c. tertiary) by
gender and age group
3. Number of students by major domains and gender
4. Number of graduates by major domains and gender
5. Trade (import and export) statistics
6. Gross Capital Formation
7. Foreign Direct Investment
8. Gini index
9. Number of households with computers / with internet access (Information Society Statistics in
general)
10. Number of persons suffering from chronic deceases by type of decease, by gender
11. Life expectance at birth (in years) by gender
12. Number of children per household (or rather Total Fertility Rate)
13. Legislators, senior officials and managers by gender
14. GVA on NACE Rev. 2 or NACE Rev 1.1 (Main Economic Sectors; e.g. Section A to U in
NACE Rev. 2)
15. Production statistics on SITC
16. Number of registered unemployed by NUTS 2 Region by gender and sex on an annual bases
from 2005 and until at least 2010;
52
17. Number of registered vacancies by Economic Sector (NACE) and Occupation (ISCO) by
NUTS 2 Region on an annual bases from 2005 and until at least 2010
18. Number of recent (e.g. past 12 months) placements by Economic Sector (NACE) and
Occupation (ISCO) by NUTS 2 Region on an annual bases from 2005 and until at least 2010
19. Expenditure on Active Labour Market Measures and Passive Measures
In general, the HDI indicators developed by United Nations serve as a platform for desirable indicators
for evaluation of HRD programmes. However, the UN HDI indicators are only available on national
level, indicating general severe limitations to its use since HRD measures always should be taken the
regional and local needs into account.
A list of general indicators for comprehensive studies of HRD programmes can be found in Annex 2 to
this report. The main part of the indicators is normally available on regional level. Please note, that the
list does not include Health Indicators and Information Society Indicators, though.
2 Demographic Indicators
The study of demographic indicators serves basically as a study of the composition of population and
population changes.
Table 2.1 shows the population by NUTS 2 Regions in Turkey in 2010 according to the Address Based
Population Registration System. The table indicates the large difference in population size across
regions of Turkey, but otherwise it is difficult to make comments to a table which shows absolute
numbers. The coloured regions of the table indicate the 12 NUTS 2 Regions being part of the Human
Resource Development Operational Programme, HRD OP.
Table 2.1 Population by NUTS 2 regions 2010
CODE
GEO
İstanbul
TR10
Tekirdağ
TR21
Balıkesir
TR22
İzmir
TR31
Aydın
TR32
Manisa
TR33
Bursa
TR41
Kocaeli
TR42
Ankara
TR51
Konya
TR52
Antalya
TR61
Adana
TR62
Hatay
TR63
Kırıkkale
TR71
Kayseri
TR72
Zonguldak
TR81
Kastamonu
TR82
Samsun
TR83
Trabzon
TR90
Erzurum
TRA1
Ağrı
TRA2
Malatya
TRB1
Van
TRB2
Gaziantep
TRC1
Şanlıurfa
TRC2
Mardin
TRC3
TURKEY TOTAL
FEMALE
6,600,591
749,613
809,478
1,963,480
1,363,692
1,472,974
1,788,340
1,617,691
2,392,490
1,134,889
1,308,782
1,875,034
1,479,406
754,854
1,165,208
517,368
374,726
1,374,337
1,269,479
527,061
549,859
814,220
992,233
1,203,683
1,593,444
986,874
36,679,806
MALE
6,655,094
771,715
833,242
1,985,368
1,375,496
1,532,584
1,807,120
1,628,456
2,379,226
1,111,589
1,376,717
1,858,090
1,525,202
741,442
1,187,763
517,703
368,303
1,366,349
1,246,688
541,385
583,801
810,938
1,030,140
1,211,150
1,598,885
998,736
37,043,182
TOTAL
13,255,685
1,521,328
1,642,720
3,948,848
2,739,188
3,005,558
3,595,460
3,246,147
4,771,716
2,246,478
2,685,499
3,733,124
3,004,608
1,496,296
2,352,971
1,035,071
743,029
2,740,686
2,516,167
1,068,446
1,133,660
1,625,158
2,022,373
2,414,833
3,192,329
1,985,610
73,722,988
Source: TÜİK (TURKSTAT) latest data available
53
Table 2.2 and table 2.3 show the average annual growth rates in population by gender (table 2.2) and
by age groups (table 2.3). The tables show no clear patterns, though some regions experience higher
than average growth rates, notably Antalya. In general, average annual growth rates are complex
measures since they contain both natural growth (stemming from births exceeding deaths) as well as
migration.
Both Gaziantep and Şanlıurfa experience above average growth rates, and both regions have an
above average growth of infant children. Otherwise, the above average growth rates of the two regions
appear to general across age and gender.
Erzurum and Ağrı have experience negative growth rates in population, which for both regions comes
from negative growth in schooling age population between 5 and 24. The rate of decrease in
population is however by no means significant.
Table 2.2 Average annual growth rates in Population by NUTS 2 regions 2007-2010
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Grand Total
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
FEMALE
1.7%
2.0%
0.8%
1.7%
1.6%
0.2%
2.1%
2.2%
2.2%
0.9%
2.8%
1.3%
1.5%
0.3%
0.6%
0.1%
0.2%
0.0%
0.3%
-0.3%
0.0%
0.7%
1.3%
2.0%
2.3%
1.3%
1.4%
MALE
1.9%
0.9%
1.2%
2.0%
1.4%
1.3%
2.4%
2.1%
2.3%
1.0%
3.3%
1.2%
2.3%
0.4%
1.1%
1.1%
0.7%
0.3%
0.4%
-0.1%
-0.2%
0.4%
0.9%
2.5%
2.4%
0.7%
1.6%
TOTAL
1.8%
1.4%
1.0%
1.9%
1.5%
0.7%
2.2%
2.2%
2.3%
0.9%
3.1%
1.2%
1.9%
0.3%
0.8%
0.6%
0.5%
0.2%
0.4%
-0.2%
-0.1%
0.5%
1.1%
2.3%
2.3%
1.0%
1.5%
Source: Own calculations based upon TÜİK (TURKSTAT) latest data available
Table 2.3 Average annual growth rates in Population by NUTS 2 regions by broad age groups 20072010
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
0-4
5-14
3.3%
2.7%
1.0%
3.2%
1.7%
0.2%
3.0%
3.1%
3.9%
0.7%
3.4%
0.4%
0.0%
-1.6%
-0.5%
-0.6%
-1.5%
0.3%
0.2%
0.8%
-0.5%
1.0%
15-24
0.2%
-1.7%
0.7%
1.2%
-0.3%
0.8%
1.7%
0.4%
0.0%
-1.1%
2.9%
25-49
2.4%
1.4%
0.3%
1.6%
1.4%
0.4%
2.1%
2.4%
2.4%
1.5%
2.9%
50-64
3.9%
5.6%
4.0%
4.2%
5.0%
3.7%
4.6%
5.0%
5.1%
3.6%
5.8%
64+
-0.2%
2.3%
2.6%
3.5%
3.2%
1.7%
2.8%
3.2%
3.5%
2.8%
3.8%
54
CODE
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
0-4
5-14
3.0%
2.6%
0.4%
0.6%
0.9%
-0.3%
-0.6%
-0.2%
-1.6%
1.5%
0.8%
0.3%
3.6%
4.3%
1.0%
2.2%
-0.7%
0.4%
-1.9%
-1.3%
-2.0%
-2.1%
-2.8%
-2.7%
-2.4%
-1.3%
-1.5%
-0.4%
0.9%
0.7%
-0.3%
-0.4%
15-24
-0.8%
0.8%
-1.2%
0.0%
-0.5%
-1.6%
-0.4%
-0.9%
0.2%
-0.8%
-1.2%
1.8%
1.3%
2.5%
1.7%
0.4%
25-49
1.5%
2.2%
0.6%
1.1%
0.3%
0.6%
0.2%
0.3%
0.3%
0.4%
1.4%
2.0%
3.0%
3.0%
2.1%
1.8%
50-64
4.5%
4.9%
3.6%
3.7%
4.0%
3.4%
2.9%
4.3%
1.7%
0.5%
3.6%
2.7%
3.7%
3.1%
1.9%
4.1%
64+
2.7%
2.2%
2.8%
2.8%
2.4%
2.1%
3.0%
2.9%
2.0%
1.2%
1.6%
1.8%
1.7%
0.5%
-0.9%
2.2%
Source: Own calculations based upon TÜİK (TURKSTAT) latest data available
Table 2.4 shows cross-regional internal migration in Turkey in percentage of total population. The
table stipulates some volume, perhaps even some high volumes here and there, but not volumes
which appear frightening in an international context. However, the table makes the reader blind in
purpose; it is obvious to look at net-migration.
Table 2.4 Cross-regional in- and out migration by NUTS 2 Regions, 2010 in pct. of total population
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ
Balıkesir
Izmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
Inmigration
3.3%
3.9%
3.1%
2.8%
2.7%
2.2%
2.9%
3.3%
3.8%
2.3%
3.6%
2.5%
2.0%
3.3%
2.8%
2.7%
4.7%
2.9%
3.6%
3.7%
2.9%
3.0%
2.3%
2.1%
2.0%
3.0%
3.0%
Outmigration
2.5%
3.1%
2.9%
2.5%
2.6%
2.7%
2.4%
2.8%
2.8%
2.7%
2.8%
2.6%
2.4%
4.7%
3.4%
3.5%
5.7%
4.1%
4.5%
4.7%
4.6%
3.7%
3.3%
2.2%
2.5%
3.5%
3.0%
Netmigration
0.8%
0.8%
0.2%
0.3%
0.2%
-0.5%
0.6%
0.5%
1.0%
-0.5%
0.8%
-0.2%
-0.4%
-1.4%
-0.6%
-0.8%
-0.9%
-1.3%
-0.9%
-1.1%
-1.7%
-0.7%
-1.1%
-0.1%
-0.5%
-0.6%
0.0%
Source: Own calculations based upon TÜİK (TURKSTAT) latest data available
It serves a clear purpose to show net-migration rates (which normally are in pro mille) since netmigration is closely linked to population increase and decrease. But net-migration is not always the
only measure of relevance in economic studies.
Table 2.5 shows basically the same, this time added internal migration within the NUTS 2 regions, and
then, as the final point, a quite interesting figure of Total mobility pressure.
55
Table 2.5 Total internal mobility pressure, provisional and cross-regional in- and out-migration, by
NUTS 2 Regions during three years (2008-2010) related to initial population (2008) in pct.
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ
Balıkesir
Izmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
In-migration
9.5%
13.4%
9.7%
9.1%
9.6%
7.8%
10.0%
11.9%
11.2%
7.4%
13.3%
8.2%
7.0%
11.4%
9.3%
9.6%
15.9%
10.2%
12.7%
10.8%
8.5%
10.2%
7.6%
7.2%
6.0%
9.6%
9.6%
Out-migration
8.1%
10.5%
8.9%
7.4%
8.6%
9.1%
7.5%
9.4%
8.6%
8.6%
10.1%
8.8%
7.9%
14.0%
11.5%
10.9%
15.5%
12.8%
13.7%
15.3%
14.9%
11.9%
11.5%
7.8%
8.0%
12.6%
9.6%
TOTAL
Mobility
Pressure
17.6%
23.8%
18.6%
16.4%
18.2%
16.9%
17.5%
21.3%
19.8%
16.1%
23.4%
17.0%
14.9%
25.3%
20.8%
20.6%
31.4%
22.9%
26.4%
26.1%
23.5%
22.1%
19.1%
15.0%
14.0%
22.2%
19.2%
Source: Own calculations based upon statistics from TÜİK (TURKSTAT) latest data available
Note: The measure is based upon data on provisional mobility on NUTS 3 level. The measure does not include external
migration (cross-national migration)
Total migration pressure is not as common as net-migration. But the point in this socio-economic study
which is explicitly addressed to HRD strategies is to show the total volume of migration within a single
as a rough estimate of the pressure on the local and regional labour market. The pressure on the local
/ regional labour market is of course that a total of 19.2 per cent of total population (which must include
labour force as well) has change location within three years21.
And that is quite impressive even on international scale indicating that the local labour market has to
cope with fast job-shifts in a quite impressive number. And that, of course, only adds to the problems
of sustainability of the local economies, and induces a significant pressure for flexibility on the work
places; the employers are faced with quite astonishing demand for continuously obtaining work
experience.
The main explanation must rely on easy access to jobs, indicating that we are talking of unskilled
labour for unskilled work. Accordingly, the indictor is another measure of the need to reinforce process
industry and advanced production. Only by increasing the demand for skilled labour and through
upgrading the educational level of the population the mobility pressure can be decreased. The outmigration regions / locations needs job opportunities and not least good and lasting jobs.
21
We further have to note that it is mostly people in the working age that moves, which makes the pressure on
the labour market even higher of course.
56
Table 2.5 shows the elderly dependency ratio while table 2.6 shows the youth dependency ratio. The
elderly dependency ratio is not something to worry about the coming years for Turkey – contrary to
many mature EU Member States. The same cannot be said about the youth dependency ratio.
Table 2.5 Elderly dependency ratio (Age 65 and over)
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
2007
8.4
12.7
16.5
11.5
13.3
13.4
11.3
10.5
9.2
11.0
10.7
8.9
9.2
11.9
11.6
13.1
22.3
14.0
16.0
11.8
8.7
11.6
6.0
8.5
6.9
7.9
2008
7.5
12.3
16.5
11.3
13.2
13.5
11.0
10.1
9.0
10.9
10.7
8.6
8.9
11.8
11.5
13.1
22.0
14.0
15.8
11.6
8.1
10.9
5.5
7.9
6.1
6.8
2009
7.7
12.5
16.8
11.6
13.4
13.6
11.2
10.3
9.2
11.2
10.8
8.8
9.0
12.1
11.7
13.4
22.5
14.4
16.1
11.9
8.5
11.3
5.7
8.0
6.3
7.0
2010
7.9
12.9
17.1
11.9
13.9
13.6
11.4
10.7
9.6
11.5
10.8
9.2
9.2
12.7
12.2
13.7
23.2
15.0
16.9
12.4
9.0
11.8
5.9
8.2
6.5
7.3
Source: TÜİK (TURKSTAT) latest data available
Table 2.6 Youth dependency ratio (Age 0-14)
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
2007
34.7
28.2
27.6
28.8
32.3
33.4
31.9
35.7
31.9
42.2
33.8
40.4
48.4
41.1
41.8
30.5
32.6
37.7
36.4
48.2
62.2
42.3
76.0
57.3
71.9
78.3
2008
34.7
27.7
27.2
28.7
31.8
33.5
31.9
35.2
32.1
41.8
34.0
40.3
48.1
40.1
41.3
30.2
31.5
36.7
34.9
47.0
60.8
41.0
72.8
57.0
70.5
74.6
2009
34.3
27.6
26.7
28.3
31.2
32.7
31.3
34.8
31.8
41.2
33.3
39.5
47.7
39.3
40.5
29.7
30.9
35.8
34.1
45.8
61.4
40.4
72.0
56.8
70.6
74.8
2010
33.8
27.6
25.9
27.6
30.8
31.3
30.7
34.4
31.4
40.6
32.3
39.2
46.9
38.8
39.5
28.7
30.4
34.7
33.4
44.4
61.4
40.1
71.2
56.1
70.0
74.2
Source: TÜİK (TURKSTAT) latest data available
57
The youth dependency ratio shows the population between 0 and 14 years of age relative to working
age population.
The remarkable feature is the very high youth dependency ratio in Agri, Van and Mardin of which the
last has a youth dependency ratio of 74 per cent. It is evident that the region cannot support itself (a
total pressure on the working age population of being able to feed and provide means to close to half
the population of the region22).
3 Main Economic Indicators
Table 3.1 shows GDP per capita (inhabitant) in Turkey and selected other countries in Europe indexed
towards the average of EU27.
Table 3.1 GDP per capita in Turkey, European Union Member States and selected other countries
selected years, indexed (Index EU27=100)
GEO
EU27
AT
BE
BG
CH
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IT
LT
LU
LV
MK
MT
NL
NO
PL
PT
RO
SE
SI
SK
TR
UK
COUNTRY
European Union (27 countries)
Austria
Belgium
Bulgaria
Switzerland
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Italy
Lithuania
Luxembourg
Latvia
FYROM
Malta
Netherlands
Norway
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
2000
100
136
129
9
198
76
31
132
171
24
82
134
124
66
0
26
145
176
110
19
264
19
10
57
138
25
65
10
159
57
21
24
143
2005
100
132
129
13
179
80
44
121
170
37
93
133
122
78
36
39
173
197
108
27
290
25
11
53
140
28
65
16
147
64
32
25
135
2006
100
131
127
15
176
80
47
119
170
42
94
133
120
80
38
38
176
185
106
30
303
30
11
54
140
30
64
19
148
65
35
26
136
2007
100
131
126
16
168
81
49
118
167
47
94
136
119
81
39
40
174
192
104
34
313
37
12
54
140
33
64
23
147
69
41
27
135
2008
100
136
129
19
179
87
57
121
170
48
95
139
121
84
43
42
162
128
105
38
324
41
13
56
145
38
65
26
144
74
48
28
118
Source: EUROSTAT and own calculations
The table clearly indicates the challenges facing Turkey in order to become a modern welfare state;
Turkey experiences a GDP/c of 28 per cent of EU average.
22
Presumably far more, since working age population consists of unemployed, students and inactive as well as
employed. The employment rate of Mardin is as low as 33 per cent as we shall see later.
58
Table 3.2 GDP/c TURKEY=100
CODE
EU27
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
European Union (27 countries)
Turkey
İstanbul
Tekirdag
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
2000
424.4
100.0
157.8
124.4
100.0
144.4
108.9
84.4
115.6
173.3
151.1
77.8
91.1
111.1
68.9
82.2
60.0
100.0
66.7
71.1
62.2
48.9
33.3
62.2
33.3
62.2
48.9
40.0
2005
401.8
100.0
175.0
133.9
94.6
132.1
101.8
82.1
142.9
150.0
139.3
66.1
103.6
78.6
62.5
64.3
67.9
121.4
73.2
67.9
60.7
48.2
42.9
55.4
35.7
57.1
42.9
41.1
2006
395.0
100.0
165.0
133.3
88.3
128.3
103.3
85.0
143.3
148.3
138.3
70.0
108.3
78.3
60.0
66.7
68.3
115.0
71.7
70.0
66.7
51.7
41.7
55.0
33.3
53.3
41.7
43.3
2007
373.1
100.0
165.7
131.3
92.5
126.9
98.5
83.6
144.8
144.8
138.8
70.1
107.5
79.1
61.2
67.2
70.1
104.5
70.1
68.7
65.7
52.2
38.8
55.2
34.3
52.2
40.3
41.8
2008
358.6
100.0
164.3
135.7
98.6
125.7
97.1
85.7
142.9
148.6
138.6
70.0
104.3
77.1
64.3
67.1
70.0
98.6
68.6
71.4
67.1
51.4
38.6
55.7
35.7
51.4
40.0
41.4
Source: EUROSTAT and own calculations
Table 3.2 the goes the level deeper and relate the GDP/c in the various NUTS 2 regions of Turkey to
relative to Turkey average. The table the again clearly indicates the divergences in Turkey. Since the
commencement of the HRD OP Samsun has catch up and taken over Konya which region is not part
of the HRD OP. Otherwise all NUTS 2 regions of the HRD OP are clearly behind the rest of the
regions of Turkey.
Table 3.3 shows share of nation Gross Value Added, GVA, by broad economic sectors across Turkish
NUTS 2 regions. Gross Value Added is a slightly more pure economic measure relative to GDP, since
it does not include taxes and subsidies.
The table does not uncover surprises though we again are able to spot some differences across
regions. Perhaps the most interesting feature of the table is the relative larger share of national GVA in
Trabzon, Hatay, Kayseri and Samsun than for instance Agri; all of these being part of the HRD OP.
Table 3.3 Share of National GVA 2008 by broad economic sectors. 2008
2008
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
Turkey
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Agriculture
100.0
0.6
3.7
5.2
3.8
6.1
8.8
4.1
4.3
2.5
5.6
Industry
100.0
26.6
3.7
1.9
6.4
2.8
4.4
10.2
8.6
7.7
1.9
Services
100.0
31.2
2.3
2.0
7.0
3.6
2.8
5.4
5.3
9.3
2.2
TOTAL
100.0
27.4
2.8
2.2
6.6
3.6
3.7
6.6
6.1
8.3
2.4
59
2008
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
Agriculture
6.6
7.2
4.8
3.5
3.4
0.9
2.0
6.3
4.5
1.7
1.8
2.2
2.6
1.9
4.2
1.7
Industry
2.1
3.5
2.7
1.4
2.6
1.8
0.5
2.3
2.1
0.6
0.3
1.0
0.6
1.8
1.1
1.3
Services
4.2
3.8
2.4
1.3
2.1
1.2
0.7
2.6
2.7
0.9
0.6
1.4
1.0
1.5
1.7
1.0
TOTAL
3.9
4.0
2.7
1.5
2.3
1.3
0.8
2.8
2.7
0.9
0.6
1.4
1.0
1.6
1.7
1.2
Source: TÜİK (TURKSTAT) latest data available and own calculations
Table 3.4 Growth rates in GVA 2005-2008 corrected for HCI
2008
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Turkey
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
Agriculture
-8.3%
-42.3%
-5.7%
-3.1%
3.3%
-8.3%
9.7%
-22.6%
-11.6%
-14.6%
-4.0%
1.8%
-8.2%
-6.5%
-18.9%
-17.1%
-12.5%
-2.2%
0.3%
-8.4%
-13.9%
-18.9%
-18.7%
-4.0%
-20.8%
-19.5%
-14.9%
Industry
14.1%
5.5%
37.7%
44.0%
0.0%
-2.8%
24.4%
19.5%
14.5%
18.2%
1.5%
5.7%
16.8%
42.6%
30.3%
26.2%
-9.7%
7.4%
25.0%
12.8%
15.9%
-4.7%
8.7%
2.6%
19.2%
12.6%
34.3%
Services
25.2%
22.9%
27.4%
34.2%
23.7%
26.2%
28.1%
25.7%
30.3%
19.2%
35.0%
18.1%
19.4%
35.0%
31.7%
23.4%
6.1%
17.7%
24.1%
29.1%
28.2%
24.3%
29.3%
30.8%
17.5%
22.8%
31.4%
TOTAL
18.5%
17.6%
25.8%
26.7%
15.5%
13.1%
22.8%
19.3%
20.7%
17.8%
17.2%
13.6%
13.7%
28.3%
17.6%
17.3%
-1.5%
10.8%
19.1%
18.8%
16.8%
6.9%
16.0%
17.0%
12.6%
9.3%
23.9%
Source: Own calculations based upon latest data available from TÜİK (TURKSTAT)
Table 3.4 then shows growth rates in GVA 2005 to 2008 corrected for inflation 23. Please note that the
measure covers a three years period and not as normal average annual growth rates. The point is of
course to indicate a potential base line for the HRD OP which also covers a three year period; we
would of course see some impact after the commencement of the HRD OP in GVA.
Otherwise, the table shows no major surprises though the differences between the regions are more
pronounced. Perhaps it is interesting that Mardin has experienced one of the highest growth rates
noting the low employment rate.
23
Table A1.1 in Annex shows Average annual growth rates in GDP/c. selected European countries for reference.
60
Table 3.5 then enters the grey zone between labour market indicators and economic indicators as it
illustrates a measure of productivity in European countries relatively to EU average. The important
note in the table is the very low productivity of Turkey, which however is increasing.
Table 3.5 Measures of Productivity: GVA per Employee, EU27=100
2005
AT
BE
BG
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IT
LT
LU
LV
MT
NL
PL
PT
RO
SE
SI
SK
TR
UK
Austria
Belgium
Bulgaria
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Italy
Lithuania
Luxembourg
Latvia
Malta
Netherlands
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
2008
121.8
135.1
14.0
76.4
40.1
117.8
136.5
35.5
90.8
120.9
128.4
85.6
42.5
41.1
156.1
121.6
27.3
294.6
24.2
59.0
119.7
32.7
58.2
17.1
128.7
57.3
32.6
35.7
122.2
127.4
139.4
17.8
82.2
53.9
116.9
141.4
45.1
98.9
129.1
132.7
93.5
51.5
47.1
155.2
NA
38.7
354.6
38.3
63.3
124.8
40.8
61.2
28.0
129.8
67.0
48.3
43.4
113.3
Note: Estimates
Source: EUROSTAT and own calculations
Table 3.6 than turn to NUTS 2 regions in Turkey; this time by main economic sectors in 2008 relative
to Turkish average. The table shows large and significant divergences in productivity across regions
where we again find Agri in the bottom.
Measures of productivity is a highly important indicator since it measure how labour intensive the
production is which is one of the best measures of how technological advanced the production is.
Obviously, the better detail in the data the better the measure; we would of course have liked if we
could have produced the measure on main NACE Rev 2.1 sectors.
Table 3.7 the shows the growth rates in productivity by main sectors between 2005 and 2008; again
on a three years scale.
Table 3.6 Measures of Productivity: GVA per Employee, by main economic sector, 2008, Turkey=100
TR
TR10
TR21
TR22
TR31
TR32
Turkey
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Agriculture
100.0
330.7
141.4
101.3
152.4
93.9
Industry
100.0
95.3
107.3
84.8
98.4
86.0
Services
100.0
143.6
86.2
85.2
120.3
90.1
TOTAL
100.0
148.4
98.4
75.1
126.2
82.2
61
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
Agriculture
97.8
147.3
213.7
306.9
172.1
154.4
190.2
104.9
107.6
107.0
38.7
117.1
57.2
37.3
39.5
53.8
95.0
65.9
127.1
155.8
112.5
Industry
101.8
104.0
156.4
131.4
94.3
84.4
83.5
73.2
112.5
85.5
129.2
94.4
85.7
106.1
168.3
103.4
101.9
91.8
56.7
79.3
133.3
Services
76.1
113.2
117.5
107.1
68.6
91.1
77.1
66.2
72.7
75.0
90.3
88.5
75.3
68.9
79.7
57.1
78.4
72.1
68.3
65.8
55.7
TOTAL
76.1
117.0
147.7
137.3
85.7
99.0
91.9
69.5
74.7
78.9
87.9
83.6
57.1
49.0
49.1
43.9
78.4
57.8
71.5
77.6
74.8
Source: Own calculations based upon latest data available from TÜİK (TURKSTAT)
Table 3.7 Measures of Productivity: Growth rates in GVA per Employee, by main economic sector,
2005 to 2008, corrected for price inflation (HCI)
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Turkey
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
Agriculture
-5.8
-38.2
14.0
15.8
32.6
24.7
38.4
-14.0
-42.1
42.4
-36.4
-36.6
-28.2
3.8
55.5
-7.9
-42.3
-42.2
4.5
4.0
18.0
-29.9
-19.9
49.8
-58.0
-16.9
7.7
Industry
6.1
6.0
9.9
15.5
4.6
6.7
12.5
19.8
-4.0
1.2
-26.2
3.5
8.6
12.9
11.4
28.9
3.6
-22.6
1.6
-19.3
-38.4
1.3
-23.2
-22.0
11.0
2.5
18.5
Services
14.9
11.1
27.9
22.9
-2.9
2.4
29.6
11.6
16.7
6.4
16.5
14.9
11.6
33.7
13.2
23.9
19.3
-4.1
18.8
23.7
-0.6
64.1
8.7
-11.6
39.6
30.8
50.6
TOTAL
12.2
11.1
22.6
26.5
4.4
15.4
31.7
15.3
0.7
5.9
-10.2
-2.8
1.8
24.6
35.0
21.8
-10.1
-23.6
16.4
20.7
23.3
5.5
0.3
15.7
0.6
12.4
39.4
Source: Own calculations based upon latest data available from TÜİK (TURKSTAT)
Finally, table 3.8 provides an overview of the main features of the main economic indicators directly
linked to labour market features as it shows the average annual growth rates in Real GVA, in
Productivity, in Real Wages, and in consumer prices.
62
Table 3.8 OVERVIEW: Table of average annual growth rates in Real GVA, Productivity, Real wages,
and consumer prices 2005-2008
GROWTH RATES
Turkey
TR
TR10 İstanbul
TR21 Tekirdağ
TR22 Balıkesir
TR31 İzmir
TR32 Aydın
TR33 Manisa
TR41 Bursa
TR42 Kocaeli
TR51 Ankara
TR52 Konya
TR61 Antalya
TR62 Adana
TR63 Hatay
TR71 Kırıkkale
TR72 Kayseri
TR81 Zonguldak
TR82 Kastamonu
TR83 Samsun
TR90 Trabzon
TRA1 Erzurum
TRA2 Ağrı
TRB1 Malatya
TRB2 Van
TRC1 Gaziantep
TRC2 Şanlıurfa
TRC3 Mardin
GVA
6.2%
5.9%
8.6%
8.9%
5.2%
4.4%
7.6%
6.4%
6.9%
5.9%
5.7%
4.5%
4.6%
9.4%
5.9%
5.8%
-0.5%
3.6%
6.4%
6.3%
5.6%
2.3%
5.3%
5.7%
4.2%
3.1%
8.0%
Productivity
4.1%
3.7%
7.5%
8.8%
1.5%
5.1%
10.6%
5.1%
0.2%
2.0%
-3.4%
-0.9%
0.6%
8.2%
11.7%
7.3%
-3.4%
-7.9%
5.5%
6.9%
7.8%
1.8%
0.1%
5.2%
0.2%
4.1%
13.1%
Wages
4.8%
5.9%
3.1%
-5.3%
-1.5%
1.5%
4.7%
5.7%
4.0%
6.7%
6.6%
6.7%
2.4%
9.4%
16.9%
8.0%
2.9%
-3.6%
-1.8%
1.4%
4.1%
43.3%
-4.5%
-0.3%
13.3%
9.7%
29.3%
HCI
10.5%
11.3%
9.6%
10.8%
9.2%
9.8%
9.9%
10.6%
11.9%
11.0%
10.2%
9.8%
10.8%
10.2%
11.0%
11.4%
10.9%
10.3%
10.8%
11.1%
11.3%
11.0%
10.4%
11.4%
10.8%
10.8%
9.8%
Source: Own calculations based upon latest data available from TÜİK (TURKSTAT)
The table tells a story by itself, comprehensive and somehow complex as it is. Showing these indictors
next to each other is not arbitrary. The main is of course to follow a causal sequence where the end
point, wage increases, never should be pursued without solid economical reason.
For instance, if GVA is growing fast we would expect a wage increase. The growth in GVA, however,
could stem from a lot of factors not least inflation. We have corrected the GVA according to very rough
measures here, but for the sake of the story we have shown the inflation rates in the last column in
any case.
The next, and most important point, is that no wage increase can be justified with increase in
productivity, which means that increase in wages has to be smaller than productivity growth in order to
ensure a sustainable development and perhaps not least a profitable economy. Recall, that GVA or
simply overhead has to finance labour costs and remuneration to business owners, but also physical
technological change and investments.
Unfortunately, no indicator which clearly can show capital formation by economic sector and region
(the more technical title of investments in physical capital) is available, recalling that investments in
Human Capital (i.e. education and training, including Lifelong Learning) is of paramount importance if
Turkey, and the Turkish regions, will catch up with the fastest growing economies, if not in Asia then in
Europe.
And then this section ends by a very clear link and indicator on HRD OP and its overall purpose. The
reason for the massive investment in Human Resource Development in Turkish regions actually has a
very serious purpose to ensure that the children have a decent opportunity to get a job.
63
4. Labour Market Indicators
Table 4.1 shows the share of population in activity according to labour force survey data. The most
interesting part is perhaps the fraction which is not in labour force. However, the share covers the
complete population which includes elderly citizens, children, and students, so perhaps the table is not
that interesting after all.
Table 4.1 Share of population in activity, total, 2010
LABOUR FORCE
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Region
Türkiye
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
Population
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
Employed
44%
42%
52%
48%
45%
52%
45%
44%
46%
42%
49%
51%
46%
43%
45%
40%
50%
54%
49%
58%
49%
44%
43%
36%
41%
31%
33%
Unemployed
6%
7%
6%
4%
8%
7%
4%
5%
7%
6%
5%
6%
9%
7%
5%
6%
6%
5%
4%
4%
4%
5%
6%
7%
6%
5%
4%
Not in
Labour
Force
49%
51%
42%
48%
46%
41%
51%
51%
47%
52%
47%
43%
45%
51%
50%
54%
44%
40%
47%
38%
47%
50%
50%
57%
53%
65%
62%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
The table however leads directly to table 4.2 which shows growth rates for the same population groups
and where we find an indicator closely linked to one of the measures of the HRD OP, namely the
growth rate of inactive24. In general, the table shows some remarkable high growth rates, and even
though the growth rates cover a three years’ span we could imagine that we, so soon in our study of
labour market indicators, have started to discover some of the fundamental problems in using LFS
data25.
24
Please note that we again, very contrary to common practice, show growth rates covering three years and not
just annual averages.
25 Basically not the purpose of the story of this study, we have note the nature of LFS Household survey which not
just are based upon self-reporting among a sample of the population, but also uses a rolling panel principle (the
respondents change every time instead of using the same). Furthermore, even though we call it Labour Force
Survey, it is basically a household survey. In general, LFS provide basic data, which is not precise enough for
creating solid interventions. However, we have no alternative in Turkey, yet
64
Table 4.2 Growth in population by activity, 2007 to 2010, total
LABOUR FORCE
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
Region
Türkiye
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
Population
5%
6%
5%
4%
6%
5%
4%
8%
7%
7%
4%
10%
4%
7%
2%
4%
3%
2%
2%
2%
1%
0%
3%
6%
8%
8%
6%
Employed
9%
3%
12%
-4%
9%
15%
5%
0%
18%
8%
19%
6%
15%
19%
11%
10%
6%
10%
-5%
3%
18%
4%
19%
12%
33%
26%
47%
Unemployed
28%
47%
45%
34%
64%
43%
13%
27%
35%
14%
-9%
69%
23%
32%
19%
32%
39%
117%
-24%
-6%
33%
89%
3%
63%
-17%
20%
-22%
Not in
Labour
Force
0%
6%
-7%
10%
-2%
-9%
2%
13%
-4%
5%
-8%
10%
-7%
-4%
-6%
-3%
-4%
-12%
14%
3%
-13%
-8%
-8%
-2%
-3%
1%
-6%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Table 4.3 then shows Economic activity rates for female across NUTS 2 regions in 2006 and 2010.
Tables for total male and female as well as other EU countries are listed in Annex 1 as tables A1.2-7.
Activity rates represent the labour force as a percentage of the population of working age (15-64
years).
Table 4.3 shows some remarkable low activity rates mainly in some the NUTS 2 regions covered by
the HRD OP. Even though Turkey on average experiences activity rates for females less than half the
EU27 average, a region like Mardin has female activity rates of 37 per cent of Turkish average; only
11 per cent of the women in Mardin are active.
Table 4.3 Economic activity rates (in per cent), Female, Turkish NUTS 2 Regions 2006 and 2010
CODE
EU27
EU25
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
GEO
European Union (27
countries)
European Union (25
countries)
Turkey
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
2006
2010
EU27=100
2006
2010
TURKEY=100
2006
2010
67.1
64.4
100.0
100.0
267.8
217.5
64.4
25.1
22.8
33.8
32.3
23.6
32.6
27.2
29.4
22.1
24.6
64.9
29.6
25.8
37.7
34.3
34.8
40.0
30.6
27.7
29.3
27.0
95.9
37.3
33.9
50.3
48.2
35.1
48.5
40.5
43.8
32.9
36.6
100.7
46.0
40.0
58.5
53.2
54.0
62.1
47.5
43.0
45.4
41.9
256.8
100.0
90.9
134.7
129.1
94.1
130.0
108.4
117.3
88.0
98.1
219.0
100.0
87.1
127.2
115.8
117.4
135.0
103.3
93.6
98.8
91.2
65
CODE
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
2006
16.5
37.0
22.9
23.9
27.6
13.9
34.6
37.7
36.1
52.0
29.1
31.5
21.9
19.6
8.6
5.8
4.9
2010
31.3
42.3
33.5
29.2
27.3
24.0
41.4
47.4
36.4
49.1
36.2
30.0
27.3
17.8
18.7
10.2
10.9
EU27=100
2006
2010
24.6
48.5
55.1
65.6
34.2
51.9
35.6
45.3
41.1
42.4
20.7
37.2
51.6
64.2
56.2
73.6
53.8
56.4
77.5
76.2
43.3
56.2
47.0
46.6
32.6
42.3
29.2
27.7
12.9
29.1
8.6
15.8
7.3
16.9
TURKEY=100
2006
2010
65.8
105.5
147.6
142.8
91.5
113.0
95.5
98.4
110.2
92.2
55.5
81.0
138.1
139.6
150.4
160.1
144.1
122.8
207.4
165.8
116.0
122.3
125.8
101.3
87.4
92.1
78.1
60.2
34.5
63.2
23.1
34.4
19.6
36.7
Source: Eurostat
Activity rates represent the labour force as a percentage of the population of working age (15-64 years).
If we then turn to youngest part of the working age population table 4.4 shows the activity rates for
female between 15 and 24 years of age. Again, the activity rates are low, though not as low as for total
women. However, this indicator bears a two-aged sword, since we would expect that a larger fraction
of the age group being heavily occupied doing studies for the purpose of obtaining a solid job on the
future labour market. And that indicates, that though Turkey appear to close in on EU27 average it
might be a sign of EU27 being better a prolonging the stay at the educational centres for female.
In any case, Turkey differentiates from the main EU Member States, but more so, the topic is of vital
interest for the HRD OP; both the topic of female / girl’s education but also the economic activity.
Table 4.4 Economic activity rates 2006 and 2010,
Turkish NUTS 2 regions, Female, Age 15-24
CODE
EU27
EU25
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
GEO
European Union (25
countries)
European Union (27
countries)
Turkey
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
2006
2010
43.5
39.7
42.3
23.4
28.7
37.1
30.6
25.7
32.8
26.5
30.0
26.3
19.5
17.7
35.5
22.9
18.6
25.1
13.7
31.3
30.2
33.7
38.4
40.7
25.5
30.2
32.5
28.5
34.0
37.6
25.3
27.8
30.1
19.3
25.5
36.9
29.7
24.1
18.7
19.6
34.6
46.2
30.6
29.5
EU27=100
2006
2010
TURKEY=100
2006
2010
100.0
100.0
185.7
155.7
97.2
53.8
66.0
85.4
70.4
59.2
75.4
61.0
68.9
60.4
44.9
40.6
81.7
52.6
42.8
57.6
31.4
72.0
69.4
77.5
88.3
102.5
64.2
76.1
81.8
71.6
85.7
94.5
63.7
70.0
75.8
48.6
64.1
92.9
74.6
60.7
47.0
49.4
87.1
116.3
76.9
74.3
180.6
100.0
122.6
158.6
130.8
109.9
140.0
113.3
127.9
112.2
83.4
75.4
151.8
97.8
79.5
107.0
58.3
133.8
128.9
144.0
164.0
159.5
100.0
118.4
127.3
111.5
133.3
147.1
99.2
108.9
118.0
75.6
99.8
144.5
116.1
94.6
73.1
76.9
135.6
181.0
119.7
115.7
66
CODE
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
2006
20.9
17.2
15.4
12.8
8.5
5.9
4.4
2010
25.5
16.0
17.1
11.7
17.3
10.8
12.1
EU27=100
2006
48.1
39.6
35.4
29.3
19.6
13.5
10.0
2010
64.1
40.4
43.0
29.4
43.4
27.2
30.3
TURKEY=100
2006
89.3
73.6
65.7
54.5
36.5
25.2
18.6
2010
99.8
62.8
67.0
45.7
67.6
42.3
47.2
Source: Eurostat
Activity rates represent the labour force as a percentage of the population of working age (15-64 years).
Table 4.5 shows employment rates by age group by Turkish NUTS 2 regions in 2010. Employment
rates are of crucial importance for the HRD OP and address issues of several Priorities. Thus table
3.4.6 shows growth rates in employment rates for male and female.
Table 4.5 indicates some differences across regions, which to some extent is a lesser surprise now
after the above studies. And notably Mardin and Sanliurfa appear to have almost criminal low
employment rates. Fortunately these two regions appear to have a high growth rate in employment
rates.
Table 4.5 Employment rates by age group 2010, male and female
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
15-24
25-54
28%
31%
34%
27%
29%
36%
24%
28%
32%
22%
34%
30%
31%
26%
26%
21%
26%
41%
30%
25%
25%
25%
23%
20%
28%
21%
21%
55-64
54%
50%
62%
60%
56%
61%
57%
55%
54%
54%
58%
61%
55%
51%
56%
51%
59%
65%
60%
70%
64%
58%
54%
47%
50%
37%
42%
28%
13%
34%
30%
20%
33%
32%
19%
28%
17%
36%
36%
28%
34%
29%
27%
46%
37%
36%
58%
40%
35%
38%
30%
26%
23%
25%
TOTAL
44%
42%
52%
48%
45%
52%
45%
44%
46%
42%
49%
51%
46%
43%
45%
40%
50%
54%
49%
58%
49%
44%
43%
36%
41%
31%
33%
Source: Own calculation based upon LFS series, TÜİK/TURKSTAT
Table 4.6 Growth in Employment rates by age group 2007 to 2010 (percentage points)
CODE
TR
TR10
TR21
TR22
TR31
TR32
GEO
Türkiye
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
15-24
-0.6%
-3.2%
0.4%
-5.0%
-4.6%
0.6%
25-54
2.3%
-1.1%
4.3%
-2.0%
3.3%
5.5%
55-64
1.2%
-0.4%
4.3%
-7.2%
2.3%
6.8%
TOTAL
1.4%
-1.6%
3.4%
-3.8%
1.0%
4.4%
67
CODE
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
15-24
-6.4%
-7.7%
4.1%
-2.2%
4.2%
-8.5%
4.6%
4.0%
1.0%
-1.7%
-6.1%
4.5%
-2.8%
-5.5%
2.7%
3.2%
7.0%
1.6%
2.0%
6.7%
10.1%
25-54
3.2%
-1.3%
3.6%
1.2%
6.8%
0.3%
4.3%
4.6%
5.2%
3.5%
4.4%
6.9%
-2.1%
1.1%
9.8%
3.2%
5.2%
3.6%
11.2%
2.9%
9.5%
55-64
4.8%
-2.4%
8.5%
1.4%
9.3%
1.1%
3.7%
5.4%
0.2%
6.6%
1.1%
-7.6%
-11.1%
4.0%
4.1%
-15.9%
2.7%
-11.4%
2.7%
2.2%
3.8%
TOTAL
0.6%
-3.3%
4.2%
0.4%
6.5%
-2.0%
4.2%
4.5%
3.5%
2.4%
1.5%
3.8%
-3.6%
0.1%
6.9%
1.7%
5.8%
1.8%
7.7%
4.3%
9.3%
Source: Own calculation based upon LFS series, TÜİK/TURKSTAT
5. Indicators on Education
Table 5.1 shows population by literacy status in 2010. Literacy is still an issue in Turkey as just 90 per
cent of the population is literate and in some of the regions as low as 82 per cent. The HRD OP has
captured that in order to increase the educational attainment in the 12 NUTS 2 regions.
Table 5.1 Population by literacy status (6 years of age and over) 2010, both gender
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
Illiterate
3.1%
3.8%
5.0%
3.8%
5.0%
6.4%
3.7%
4.2%
3.0%
5.2%
3.0%
5.9%
7.2%
6.7%
6.3%
7.4%
9.4%
7.6%
8.4%
8.3%
10.2%
9.2%
11.7%
7.9%
12.2%
12.2%
5.8%
Literate
91.5%
93.2%
91.7%
92.1%
91.2%
91.4%
92.8%
92.1%
92.9%
92.2%
92.9%
90.3%
89.0%
90.2%
88.4%
90.0%
87.9%
88.7%
87.7%
87.1%
81.7%
86.4%
83.1%
87.8%
82.3%
83.3%
90.0%
Unknown
5.3%
3.0%
3.3%
4.2%
3.8%
2.2%
3.5%
3.8%
4.1%
2.5%
4.1%
3.7%
3.8%
3.1%
5.3%
2.7%
2.6%
3.8%
3.9%
4.6%
8.1%
4.4%
5.2%
4.2%
5.5%
4.5%
4.2%
TOTAL
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
Source: TÜİK/TURKSTAT and own calculations
68
Table 5.2 shows the same for the female population, which issue is of particular focus of one the five
Priorities of the HRD OP. Again, Turkey has a low literacy level and in some regions even as low as
75 per cent of the total female population cannot read and write.
Table 5.2 Population by literacy status (6 years of age and over) 2010, female
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
Illiterate
5.2%
6.1%
7.8%
6.1%
8.3%
10.6%
6.1%
7.0%
5.1%
8.8%
5.2%
9.6%
11.8%
11.3%
10.4%
12.2%
14.6%
11.7%
13.9%
13.8%
16.8%
14.6%
19.0%
12.9%
19.7%
19.6%
9.5%
Literate
89.9%
91.3%
89.3%
90.2%
88.3%
87.7%
90.6%
89.3%
91.0%
89.0%
91.3%
87.0%
84.9%
85.8%
84.3%
85.2%
83.0%
84.7%
82.2%
81.6%
75.2%
80.9%
75.4%
83.1%
74.2%
75.5%
86.5%
Unknown
4.9%
2.6%
2.9%
3.7%
3.3%
1.8%
3.3%
3.7%
3.9%
2.3%
3.5%
3.4%
3.3%
2.9%
5.3%
2.6%
2.4%
3.6%
3.9%
4.6%
8.0%
4.4%
5.6%
4.0%
6.2%
4.9%
4.0%
Source: TÜİK/TURKSTAT and own calculations
Table 5.3 are able to split literacy up according to educational attainment for male and female in 2010.
The main regional divergences can be found in column 1 and 2 on illiteracy (which we have dealt with
just above) and “Literate but no school completed”, where some of the regions of the HRD OP show
very high rates.
Table 5.3 Share of Population aged 15 and over by educational attainment, 2010, Male and Female
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Illiterate
3.7%
4.4%
5.7%
4.3%
5.9%
7.4%
4.3%
5.0%
3.5%
6.4%
3.6%
7.2%
Literate but
no school
completed
4.3%
4.3%
5.1%
4.1%
4.5%
5.2%
3.8%
4.3%
2.9%
3.9%
5.2%
5.8%
Junior high
school or
vocational
school at
junior high
school level
graduate
6.7%
6.5%
5.3%
6.4%
5.1%
4.9%
7.2%
6.3%
7.2%
4.8%
5.5%
6.0%
High school
or
vocational
school at
high school
level
graduate
23.4%
22.6%
19.4%
23.2%
18.9%
17.9%
23.9%
22.7%
27.4%
17.2%
21.9%
21.8%
Higher
education
graduate
and more
11.8%
8.5%
8.9%
12.1%
9.1%
6.8%
9.7%
8.8%
17.0%
7.8%
10.2%
8.8%
69
CODE
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
Illiterate
9.1%
8.1%
7.6%
8.5%
10.9%
9.0%
9.9%
10.3%
13.7%
11.1%
16.2%
10.3%
16.7%
17.1%
7.0%
Literate but
no school
completed
6.3%
4.9%
4.9%
5.6%
7.1%
6.3%
6.2%
7.6%
13.6%
7.2%
16.2%
7.9%
16.4%
13.6%
5.9%
Junior high
school or
vocational
school at
junior high
school level
graduate
5.4%
6.1%
5.5%
5.3%
4.9%
4.7%
5.7%
4.9%
3.4%
5.7%
2.8%
4.8%
2.8%
3.0%
5.7%
High school
or
vocational
school at
high school
level
graduate
18.6%
19.3%
20.7%
20.0%
16.3%
17.5%
21.0%
20.2%
13.5%
21.3%
13.7%
16.0%
13.2%
15.0%
20.8%
Higher
education
graduate
and more
6.5%
7.1%
7.8%
7.4%
6.5%
7.2%
7.8%
7.5%
4.2%
7.8%
4.0%
5.6%
4.3%
4.2%
9.2%
Source: TÜİK/TURKSTAT and own calculations
See more extensive table in Annex 1
Table 5.4, showing Share of population age 15 and above with at least High school or vocational
school at high school level, tries to come closer to a doable international measure (we often tries to
obtain a measure of share of persons aged 25 with at least ISCED 3). The table are able to indicate a
significant gender difference, which difference becomes more accentuate when we look at the NUTS 2
regions taking part of the HRD OP.
Table 5.4 Share of population age 15 and above with at least High school or vocational school at high
school level, male and female, 2008 and 2010
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ
Balıkesir
İzmir
Aydın
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kırıkkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Ağrı
Malatya
Van
Gaziantep
Şanlıurfa
Mardin
TURKEY
MALE
33.7%
31.2%
29.1%
34.0%
27.1%
25.8%
34.7%
32.7%
45.5%
28.5%
31.4%
30.4%
26.1%
28.5%
30.5%
29.1%
25.2%
25.7%
30.4%
31.4%
20.0%
32.8%
21.7%
23.1%
20.3%
24.8%
30.7%
2008
FEMALE
28.4%
23.2%
20.0%
28.7%
21.2%
16.0%
24.0%
21.3%
36.4%
16.3%
24.0%
23.4%
17.0%
16.8%
18.7%
17.9%
14.1%
16.4%
19.2%
15.5%
9.9%
18.4%
8.1%
13.5%
9.2%
9.0%
21.7%
TOTAL
31.0%
27.4%
24.5%
31.3%
24.2%
20.8%
29.3%
27.0%
40.9%
22.3%
27.7%
26.8%
21.5%
22.6%
24.6%
23.4%
19.5%
21.0%
24.8%
23.5%
15.2%
25.6%
15.1%
18.2%
14.7%
17.1%
26.2%
MALE
38.1%
35.2%
33.3%
38.3%
31.1%
30.4%
39.4%
37.7%
49.2%
31.6%
36.1%
34.6%
30.2%
32.7%
34.9%
33.9%
29.2%
30.1%
35.1%
35.9%
22.8%
36.7%
25.0%
26.9%
23.8%
27.7%
35.0%
2010
FEMALE
32.2%
26.8%
23.2%
32.3%
24.9%
18.8%
27.8%
25.3%
39.7%
18.7%
27.9%
26.5%
19.8%
20.2%
22.1%
21.0%
16.8%
19.5%
22.6%
19.4%
12.3%
21.7%
10.2%
16.3%
11.5%
10.9%
25.1%
TOTAL
35.2%
31.1%
28.3%
35.3%
28.0%
24.7%
33.6%
31.5%
44.4%
25.0%
32.1%
30.5%
25.1%
26.3%
28.5%
27.4%
22.9%
24.8%
28.7%
27.7%
17.7%
29.1%
17.7%
21.6%
17.5%
19.3%
30.0%
Source: TUIK/TURKSTAT and own calculations
70
Table 5.5 finally shows the growth in the share of population with at least High school or vocational
school at high school level where the most remarkable feature is the regions experiencing a lower than
average growth rate. That is the feature of most of the NUTS 2 Regions of HRD OP.
Table 5.5 Share of population age 15 and above with at least High school or vocational school at high
school level, male and female, 2008 and 2010: Growth from 2008 to 2010 in percentage points
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
TURKEY
MALE
4.4%
4.0%
4.2%
4.3%
4.0%
4.6%
4.7%
5.0%
3.7%
3.1%
4.7%
4.2%
4.1%
4.2%
4.4%
4.7%
4.0%
4.4%
4.7%
4.5%
2.8%
3.9%
3.3%
3.9%
3.4%
2.9%
4.2%
FEMALE
3.9%
3.6%
3.2%
3.6%
3.7%
2.7%
3.8%
4.0%
3.4%
2.4%
3.9%
3.1%
2.8%
3.4%
3.4%
3.0%
2.7%
3.1%
3.4%
3.9%
2.4%
3.3%
2.1%
2.8%
2.3%
1.9%
3.4%
TOTAL
4.2%
3.7%
3.7%
4.0%
3.8%
3.8%
4.3%
4.5%
3.5%
2.8%
4.4%
3.7%
3.6%
3.7%
3.9%
4.0%
3.4%
3.8%
4.0%
4.2%
2.5%
3.5%
2.6%
3.3%
2.8%
2.2%
3.8%
Source: TÜİK/TURKSTAT and own calculations
Something appears odd, however; in just three years the share of population with at least High School
level has increased significantly in all regions by 3.8 per cent on average. The significant increase
does call for a quality check of the data, which however was not possible during this study.
The Socio-Economic Study is not designated to provide basic new knowledge inasmuch as the Interim
Evaluation would anticipate that the table and the results shown are well-known to all stakeholders in
the HRD OP. The study is solely meant as a reference for the remainder of the Interim Evaluation
report and does accordingly not call for conclusions.
71
Appendix 1. Background Tables
Table A1.1 Average annual growth rates in GDP/c. selected European countries
GEO
COUNTRY
2005
2006
EU27
AT
BE
BG
CH
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IT
LT
LU
LV
MK
MT
NL
NO
PL
PT
RO
SE
SI
SK
TR
UK
European Union (27 countries)
Austria
Belgium
Bulgaria
Switzerland
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Italy
Lithuania
Luxembourg
Latvia
FYROM
Malta
Netherlands
Norway
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
3.6
2.9
3.5
15.3
1.4
4.8
12.7
1.7
3.6
16.9
6.6
3.5
3.1
7.8
NA
15.2
8.3
6.5
3.3
NA
14.9
5.9
11.7
5.3
2.0
4.0
5.8
6.1
3.5
21.1
1.9
6.7
14.6
5.3
5.1
4.5
13.3
3.5
5.6
13.3
3.7
5.0
20.5
6.7
5.0
4.0
8.6
9.9
1.1
6.7
-1.4
3.3
NA
16.4
10.1
22.8
8.3
6.7
5.1
9.5
10.9
3.4
21.6
6.1
7.6
16.9
2007
2008
5.5
5.5
4.3
17.6
0.7
6.8
10.8
5.0
3.7
18.0
5.4
7.9
4.2
6.8
10.1
12.4
4.3
9.6
3.2
NA
19.7
8.8
32.9
11.5
6.3
5.4
4.5
15.5
5.3
28.9
5.1
10.3
22.9
0.4
3.7
2.2
17.5
6.7
7.4
15.4
2.0
1.9
1.7
1.7
2.4
2.4
3.9
9.2
6.0
-6.7
-32.9
0.8
NA
12.9
4.0
9.7
13.8
4.4
4.0
6.6
15.9
1.9
12.1
-2.2
7.6
16.7
Source: EUROSTAT and own calculations
72
Table A1.2 Economic activity rates (in per cent), both gender
CODE
EU27
EU25
AT
BE
BG
CH
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IS
IT
LT
LU
LV
MT
NL
NO
PL
PT
RO
SE
SI
SK
TR
UK
GEO
European Union (27 countries)
European Union (25 countries)
Austria
Belgium
Bulgaria
Switzerland
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Iceland
Italy
Lithuania
Luxembourg
Latvia
Malta
Netherlands
Norway
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
2006
71.0
71.4
75.1
67.7
66.5
82.4
74.4
70.2
76.6
79.5
73.8
73.4
74.5
70.3
68.2
61.5
62.4
69.5
84.7
62.2
70.5
68.2
73.2
60.2
78.2
78.1
65.6
74.0
63.7
79.5
71.5
68.7
51.9
75.5
2010
75.0
71.9
73.7
66.5
64.5
81.3
73.0
70.3
75.3
0.0
72.4
70.9
75.2
69.6
67.0
0.0
62.0
71.8
87.1
62.7
67.4
66.7
71.3
57.6
77.4
78.0
63.3
73.9
0.0
0.0
70.9
68.6
49.0
75.7
EU27 AVERAGE
2006
2010
100.0
100.0
100.6
96.0
105.7
98.3
95.3
88.7
93.7
86.0
116.1
108.4
104.8
97.4
98.8
93.8
107.9
100.4
111.9
0.0
103.9
96.7
103.3
94.5
104.9
100.3
99.1
92.9
96.1
89.4
86.6
0.0
87.9
82.7
97.9
95.8
119.3
116.2
87.6
83.7
99.3
90.0
96.1
89.0
103.1
95.1
84.8
76.9
110.1
103.2
110.0
104.1
92.4
84.4
104.2
98.6
89.6
0.0
112.0
0.0
100.7
94.6
96.7
91.6
73.1
65.3
106.3
101.0
TURKEY
AVERAGE
2006
2010
136.7
153.1
137.5
146.9
144.6
150.5
130.3
135.8
128.1
131.6
158.7
165.9
143.3
149.1
135.1
143.6
147.5
153.7
153.0
0.0
142.0
147.9
141.2
144.7
143.4
153.6
135.4
142.1
131.4
136.9
118.4
0.0
120.2
126.5
133.8
146.6
163.1
177.8
119.7
128.1
135.8
137.7
131.3
136.2
140.9
145.5
116.0
117.6
150.5
158.0
150.4
159.4
126.4
129.2
142.5
150.9
122.5
0.0
153.1
0.0
137.7
144.8
132.2
140.1
100.0
100.0
145.3
154.6
Source: EUROSTAT and own calculations
73
Table A1.3 Economic activity rates (in per cent), female
CODE
EU27
EU25
AT
BE
BG
CH
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IS
IT
LT
LU
LV
MT
NL
NO
PL
PT
RO
SE
SI
SK
TR
UK
GEO
European Union (27 countries)
European Union (25 countries)
Austria
Belgium
Bulgaria
Switzerland
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Iceland
Italy
Lithuania
Luxembourg
Latvia
Malta
Netherlands
Norway
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
2006
67.1
64.35
66.99
59.53
60.23
74.71
63.8
62.34
69.1
0
69.31
60.17
73.26
64.62
55.01
0
55.51
61.9
83.37
50.81
64.57
58.23
66.65
36.52
70.73
74.75
56.7
68.43
0
0
66.74
60.93
25.06
69.24
2010
64.43
64.88
69.33
61.82
62.32
76.43
67.39
61.52
70.82
76.12
70.99
65.9
72.5
66.02
57.55
55.89
56.7
61.97
81.81
51.13
68.8
60.28
70.72
42.19
72.61
75.52
59.01
69.86
55.79
76.69
67.4
61.31
29.62
69.38
EU27 AVERAGE
2006
2010
100.0
100.0
95.9
100.7
99.8
107.6
88.7
95.9
89.8
96.7
111.3
118.6
95.1
104.6
92.9
95.5
103.0
109.9
0.0
118.1
103.3
110.2
89.7
102.3
109.2
112.5
96.3
102.5
82.0
89.3
0.0
86.7
82.7
88.0
92.3
96.2
124.2
127.0
75.7
79.4
96.2
106.8
86.8
93.6
99.3
109.8
54.4
65.5
105.4
112.7
111.4
117.2
84.5
91.6
102.0
108.4
0.0
86.6
0.0
119.0
99.5
104.6
90.8
95.2
37.3
46.0
103.2
107.7
TURKEY
AVERAGE
2006
2010
267.8
217.5
256.8
219.0
267.3
234.1
237.5
208.7
240.3
210.4
298.1
258.0
254.6
227.5
248.8
207.7
275.7
239.1
0.0
257.0
276.6
239.7
240.1
222.5
292.3
244.8
257.9
222.9
219.5
194.3
0.0
188.7
221.5
191.4
247.0
209.2
332.7
276.2
202.8
172.6
257.7
232.3
232.4
203.5
266.0
238.8
145.7
142.4
282.2
245.1
298.3
255.0
226.3
199.2
273.1
235.9
0.0
188.4
0.0
258.9
266.3
227.5
243.1
207.0
100.0
100.0
276.3
234.2
Source: Eurostat
Activity rates represent the labour force as a percentage of the population of working age (15-64 years).
74
Table A1.4 Economic activity rates at NUTS levels 2 (in per cent), male
CODE
EU25
EU27
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
European Union (25
countries)
European Union (27
countries)
Turkey
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
EU27 AVERAGE
2006
2010
TURKEY AVERAGE
2006
2010
2006
2010
71.9
71.4
96.0
100.6
146.9
137.5
75.0
49.0
49.4
58.4
53.5
47.8
53.7
50.7
52.7
48.3
48.3
44.2
59.3
47.7
45.3
49.7
41.4
53.4
58.6
54.5
63.9
50.3
52.6
43.5
42.6
40.6
32.6
30.4
71.0
51.9
50.8
58.7
53.6
54.6
58.9
53.1
50.6
52.4
49.5
54.7
61.6
55.3
51.4
50.0
48.1
57.3
62.2
54.6
61.8
56.3
52.9
49.6
45.1
47.3
34.5
37.8
100.0
65.3
65.9
77.9
71.4
63.7
71.6
67.7
70.4
64.4
64.5
58.9
79.1
63.6
60.4
66.2
55.2
71.2
78.2
72.8
85.3
67.1
70.1
58.0
56.9
54.1
43.5
40.5
100.0
73.1
71.5
82.7
75.5
76.9
82.9
74.8
71.2
73.8
69.7
77.0
86.7
77.9
72.3
70.5
67.7
80.6
87.6
76.9
87.0
79.3
74.5
69.9
63.5
66.6
48.6
53.3
153.1
100.0
100.9
119.3
109.3
97.6
109.6
103.6
107.7
98.6
98.7
90.2
121.0
97.4
92.4
101.4
84.5
109.0
119.7
111.4
130.5
102.7
107.3
88.8
87.0
82.8
66.5
62.0
136.7
100.0
97.7
113.1
103.2
105.2
113.4
102.2
97.4
101.0
95.2
105.2
118.5
106.5
98.9
96.3
92.6
110.2
119.7
105.1
118.9
108.4
101.9
95.6
86.8
91.0
66.5
72.8
Source: EUROSTAT, Labour Force Statistics
75
Table A1.5 Lowest 15 Economic activity rates at NUTS levels 2 (in per cent) among 319 identified
NUTS 2 Regions in EUROSTAT, Sorted by EU27 Average 2006
CODE
TRA1
TR71
TR10
TR51
TR42
TR31
TR62
TR63
TR52
TRB1
TRB2
TR72
TRC1
TRC2
TRC3
GEO
Erzurum
Kirikkale
Istanbul
Ankara
Kocaeli
Izmir
Adana
Hatay
Konya
Malatya
Van
Kayseri
Gaziantep
Sanliurfa
Mardin
Sorted by EU27 Average 2010
ITF4
Puglia
TR63
Hatay
TR10
Istanbul
TR41
Bursa
ITG1
Sicilia
TR71
Kirikkale
TRB1
Malatya
TR51
Ankara
TR72
Kayseri
ITF6
Calabria
TRC1
Gaziantep
ITF3
Campania
TRB2
Van
TRC3
Mardin
TRC2
Sanliurfa
2006
50.28
49.65
49.39
48.34
48.29
47.78
47.69
45.26
44.15
43.48
42.62
41.37
40.57
32.57
30.35
2010
56.29
50.04
50.76
49.46
52.44
54.63
55.34
51.37
54.65
49.64
45.09
48.08
47.27
34.54
37.83
52.48
45.26
49.39
52.73
52.12
49.65
43.48
48.34
41.37
52.38
40.57
50.7
42.62
30.35
32.57
51.41
51.37
50.76
50.59
50.06
50.04
49.64
49.46
48.08
47.94
47.27
46.44
45.09
37.83
34.54
EU27 AVERAGE
2006a
2010b
67.1
79.3
66.2
70.5
65.9
71.5
64.5
69.7
64.4
73.8
63.7
76.9
63.6
77.9
60.4
72.3
58.9
77.0
58.0
69.9
56.9
63.5
55.2
67.7
54.1
66.6
43.5
48.6
40.5
53.3
70.0
60.4
65.9
70.4
69.5
66.2
58.0
64.5
55.2
69.9
54.1
67.6
56.9
40.5
43.5
72.4
72.3
71.5
71.2
70.5
70.5
69.9
69.7
67.7
67.5
66.6
65.4
63.5
53.3
48.6
Source: EUROSTAT and own calculations
76
Table A1.6 Economic activity rates 2006 and 2010, both gender, Age 15-24
CODE
EU27
EU25
AT
BE
BG
CH
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IS
IT
LT
LU
LV
MT
NL
NO
PL
PT
RO
SE
SI
SK
TR
UK
GEO
European Union (27 countries)
European Union (25 countries)
Austria
Belgium
Bulgaria
Switzerland
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Iceland
Italy
Lithuania
Luxembourg
Latvia
Malta
Netherlands
Norway
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
2006
47.3
45.8
59.4
34.7
28.9
68.6
41.5
33.5
50.2
0.0
35.9
48.2
51.8
38.2
32.4
0.0
26.8
54.7
78.6
32.5
26.3
27.8
40.8
52.6
70.8
57.4
34.0
42.7
0.0
0.0
40.6
35.4
36.3
62.5
2010
43.0
43.9
58.8
32.5
28.9
67.9
40.6
30.9
51.3
67.4
38.3
42.7
49.4
39.4
30.3
37.7
24.9
42.0
73.7
28.4
29.6
24.7
40.4
51.5
69.0
56.7
34.5
36.7
31.2
51.7
39.9
31.1
37.4
59.2
EU27=100
2006
100.0
96.9
125.6
73.4
61.0
145.1
87.8
70.9
106.1
0.0
76.0
101.8
109.5
80.7
68.5
0.0
56.6
115.6
166.2
68.7
55.5
58.8
86.3
111.1
149.7
121.4
71.8
90.3
0.0
0.0
85.9
74.7
76.7
132.1
2010
100.0
102.0
136.9
75.6
67.3
157.9
94.4
71.8
119.4
156.7
89.1
99.3
114.9
91.6
70.6
87.7
58.0
97.8
171.4
66.0
68.9
57.5
94.0
119.9
160.5
131.9
80.2
85.4
72.7
120.3
92.9
72.3
86.9
137.6
TURKEY=100
2006
2010
130.4
115.0
126.4
117.4
163.9
157.5
95.7
87.0
79.6
77.4
189.2
181.6
114.4
108.6
92.5
82.6
138.4
137.4
0.0
180.3
99.1
102.5
132.8
114.2
142.8
132.1
105.2
105.4
89.3
81.2
0.0
100.9
73.8
66.7
150.7
112.5
216.8
197.1
89.6
76.0
72.4
79.3
76.7
66.2
112.6
108.1
144.9
137.9
195.2
184.7
158.4
151.8
93.7
92.2
117.8
98.3
0.0
83.6
0.0
138.4
112.0
106.9
97.5
83.1
100.0
100.0
172.3
158.3
Source: EUROSTAT and own calculations
77
Table A1.7 Economic activity rates 2006 and 2010, Female, Age 15-24
CODE
EU27
EU25
AT
BE
BG
CH
CY
CZ
DE
DK
EE
ES
FI
FR
GR
HR
HU
IE
IS
IT
LT
LU
LV
MT
NL
NO
PL
PT
RO
SE
SI
SK
TR
UK
GEO
European Union (27 countries)
European Union (25 countries)
Austria
Belgium
Bulgaria
Switzerland
Cyprus
Czech Republic
Germany
Denmark
Estonia
Spain
Finland
France
Greece
Croatia
Hungary
Ireland
Iceland
Italy
Lithuania
Luxembourg
Latvia
Malta
Netherlands
Norway
Poland
Portugal
Romania
Sweden
Slovenia
Slovakia
Turkey
United Kingdom
2006
43.5
42.3
55.1
31.9
26.4
67.0
38.4
29.2
47.6
0.0
30.6
43.9
51.0
34.4
28.7
0.0
23.4
50.2
80.3
27.0
23.1
25.0
33.6
48.3
70.1
58.0
30.5
38.7
0.0
0.0
36.4
30.9
23.4
59.7
2010
39.7
40.7
54.1
29.8
24.2
66.5
40.7
25.3
48.9
67.2
34.3
40.1
49.3
35.8
27.2
31.2
22.1
41.2
76.1
23.4
26.3
22.7
37.7
47.4
69.4
56.9
29.7
34.8
26.1
51.4
34.8
25.5
25.5
56.4
EU27=100
2006
100.0
97.2
126.7
73.4
60.6
154.2
88.2
67.1
109.4
0.0
70.4
101.0
117.4
79.0
65.9
0.0
53.8
115.5
184.7
62.0
53.1
57.5
77.4
111.0
161.1
133.3
70.1
89.0
0.0
0.0
83.7
71.0
53.8
137.4
2010
100.0
102.5
136.2
74.9
60.8
167.4
102.5
63.6
123.0
169.2
86.2
101.0
124.1
90.0
68.5
78.5
55.6
103.7
191.5
58.8
66.2
57.0
94.8
119.3
174.7
143.1
74.7
87.5
65.7
129.2
87.6
64.2
64.2
141.9
TURKEY=100
2006
2010
185.7
155.7
180.6
159.5
235.2
212.0
136.4
116.6
112.6
94.7
286.3
260.6
163.8
159.6
124.6
98.9
203.1
191.4
0.0
263.3
130.7
134.2
187.7
157.2
218.0
193.2
146.7
140.0
122.4
106.7
0.0
122.2
99.9
86.5
214.6
161.5
343.1
298.0
115.1
91.5
98.5
103.1
106.8
88.7
143.7
147.6
206.2
185.7
299.2
272.0
247.6
222.7
130.2
116.3
165.4
136.2
0.0
102.3
0.0
201.2
155.5
136.4
131.8
99.9
100.0
100.0
255.2
220.9
Source: EUROSTAT and own calculations
78
Table A1.8 Share of population in activity, 2010, total population
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
44%
6%
49%
42%
7%
51%
52%
6%
42%
48%
4%
48%
45%
8%
46%
52%
7%
41%
45%
4%
51%
44%
5%
51%
46%
7%
47%
42%
6%
52%
49%
5%
47%
51%
6%
43%
46%
9%
45%
43%
7%
51%
45%
5%
50%
40%
6%
54%
50%
6%
44%
54%
5%
40%
49%
4%
47%
58%
4%
38%
49%
4%
47%
44%
5%
50%
43%
6%
50%
36%
7%
57%
41%
6%
53%
31%
5%
65%
33%
4%
62%
MALE
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
63%
8%
63%
10%
71%
7%
63%
5%
62%
9%
68%
8%
62%
5%
64%
7%
66%
9%
62%
7%
70%
6%
65%
7%
65%
12%
58%
10%
64%
8%
59%
8%
62%
9%
68%
5%
64%
5%
69%
6%
64%
5%
58%
9%
62%
9%
55%
12%
65%
10%
51%
9%
56%
8%
FEMALE
NILF
29%
27%
23%
32%
29%
24%
33%
29%
25%
31%
23%
28%
24%
32%
28%
32%
29%
27%
31%
26%
30%
32%
29%
32%
25%
40%
36%
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
26%
4%
21%
4%
33%
5%
31%
3%
28%
7%
35%
6%
27%
2%
24%
3%
25%
5%
22%
4%
28%
3%
36%
5%
27%
7%
26%
4%
25%
2%
19%
4%
37%
4%
41%
5%
34%
3%
46%
2%
34%
1%
29%
1%
25%
4%
15%
3%
17%
2%
10%
1%
10%
1%
NILF
71%
75%
62%
66%
65%
59%
70%
73%
70%
73%
69%
59%
66%
70%
72%
76%
59%
53%
63%
51%
65%
70%
72%
82%
82%
89%
89%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
79
Table A1.9 Share of population in activity, 2010, Age 15 to 24
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
28%
8%
65%
31%
9%
59%
34%
9%
57%
27%
5%
68%
29%
11%
59%
36%
10%
54%
24%
5%
71%
28%
6%
65%
32%
10%
58%
22%
8%
71%
34%
6%
59%
30%
8%
62%
31%
12%
57%
26%
8%
66%
26%
6%
68%
21%
7%
72%
26%
10%
64%
41%
7%
52%
30%
4%
65%
25%
6%
69%
25%
4%
71%
25%
5%
70%
23%
8%
69%
20%
8%
72%
28%
6%
66%
21%
4%
75%
21%
5%
74%
MALE
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
35%
9%
40%
11%
43%
11%
31%
5%
34%
12%
44%
11%
28%
6%
36%
8%
41%
12%
30%
8%
48%
8%
32%
9%
43%
14%
33%
10%
38%
9%
28%
9%
27%
12%
43%
7%
33%
5%
27%
9%
28%
6%
33%
8%
31%
11%
32%
12%
41%
10%
32%
8%
31%
8%
FEMALE
NILF
88%
97%
94%
72%
85%
91%
64%
84%
99%
88%
95%
72%
98%
79%
92%
77%
81%
86%
78%
93%
69%
77%
94%
86%
97%
101%
90%
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
20%
6%
23%
8%
23%
6%
22%
4%
23%
11%
28%
10%
19%
4%
20%
5%
22%
9%
12%
7%
21%
5%
26%
7%
20%
9%
19%
5%
15%
4%
13%
5%
25%
8%
38%
9%
27%
4%
23%
3%
22%
2%
15%
2%
15%
5%
8%
3%
14%
2%
11%
1%
11%
1%
NILF
75%
69%
70%
73%
66%
62%
76%
75%
69%
81%
74%
67%
70%
76%
81%
82%
68%
52%
69%
73%
76%
83%
80%
88%
83%
88%
88%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
80
Table A1.10 Share of population in activity, 2010, Age 25 to 54
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
54%
6%
39%
50%
7%
43%
62%
6%
32%
60%
5%
36%
56%
8%
35%
61%
7%
32%
57%
4%
39%
55%
5%
40%
54%
7%
39%
54%
6%
40%
58%
4%
38%
61%
6%
33%
55%
10%
36%
51%
7%
41%
56%
5%
39%
51%
7%
42%
59%
6%
34%
65%
6%
29%
60%
5%
36%
70%
4%
25%
64%
3%
32%
58%
6%
36%
54%
6%
40%
47%
8%
45%
50%
6%
44%
37%
6%
57%
42%
5%
53%
MALE
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
78%
9%
76%
10%
84%
6%
82%
6%
78%
10%
80%
8%
81%
6%
81%
8%
79%
9%
79%
8%
83%
6%
81%
7%
78%
12%
72%
11%
79%
8%
77%
9%
78%
9%
84%
6%
81%
6%
85%
6%
87%
5%
74%
11%
78%
9%
72%
13%
80%
10%
65%
10%
74%
9%
FEMALE
NILF
13%
14%
10%
12%
12%
12%
14%
11%
11%
13%
11%
12%
10%
18%
12%
14%
13%
10%
13%
9%
7%
15%
13%
15%
10%
25%
17%
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
30%
4%
23%
4%
39%
5%
37%
3%
34%
7%
41%
7%
32%
2%
29%
3%
28%
5%
29%
4%
32%
3%
40%
5%
31%
7%
30%
4%
31%
3%
23%
5%
41%
4%
46%
5%
39%
3%
55%
2%
40%
1%
40%
1%
29%
3%
20%
2%
19%
2%
10%
1%
11%
1%
NILF
66%
74%
55%
60%
59%
53%
66%
68%
68%
67%
64%
54%
62%
66%
66%
72%
56%
49%
58%
42%
59%
60%
68%
78%
80%
90%
89%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
81
Table A1.11 Share of population in activity, 2010, Age 55 to 64
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
28%
2%
71%
13%
2%
85%
34%
2%
64%
30%
1%
69%
20%
3%
77%
33%
2%
65%
32%
1%
67%
19%
1%
81%
28%
2%
70%
17%
2%
82%
36%
2%
62%
36%
3%
61%
28%
3%
69%
34%
2%
63%
29%
2%
69%
27%
2%
71%
46%
1%
53%
37%
2%
61%
36%
0%
63%
58%
0%
42%
40%
3%
57%
35%
2%
64%
38%
2%
61%
30%
2%
68%
26%
2%
72%
23%
1%
77%
25%
1%
74%
MALE
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
40%
3%
23%
4%
49%
4%
40%
2%
32%
4%
48%
3%
47%
1%
30%
1%
43%
3%
27%
3%
54%
2%
46%
5%
38%
6%
48%
4%
42%
2%
41%
3%
53%
2%
44%
2%
47%
1%
67%
1%
48%
3%
52%
3%
49%
4%
49%
0%
41%
5%
41%
3%
49%
0%
FEMALE
NILF
57%
73%
47%
58%
64%
49%
51%
68%
54%
70%
43%
50%
56%
48%
57%
56%
45%
54%
52%
32%
49%
44%
47%
51%
54%
56%
51%
POP
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
100%
LABOUR FORCE
EMP
UEMP
16%
0%
3%
0%
18%
0%
21%
0%
9%
0%
18%
0%
19%
0%
7%
0%
15%
0%
6%
0%
20%
0%
27%
1%
17%
1%
21%
0%
18%
0%
13%
0%
39%
0%
31%
2%
25%
0%
48%
0%
36%
0%
19%
0%
25%
0%
14%
2%
11%
0%
7%
0%
5%
0%
NILF
84%
96%
81%
79%
91%
82%
82%
93%
86%
94%
81%
73%
82%
79%
83%
87%
61%
67%
75%
52%
64%
81%
73%
84%
89%
93%
95%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
82
Table A1.12 Growth in population by activity, 2007 to 2010, Total
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
5%
6%
5%
4%
6%
5%
4%
8%
7%
7%
4%
10%
4%
7%
2%
4%
3%
2%
2%
2%
1%
0%
3%
6%
8%
8%
6%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
9%
28%
0%
3%
47%
6%
12%
45%
-7%
-4%
34%
10%
9%
64%
-2%
15%
43%
-9%
5%
13%
2%
0%
27%
13%
18%
35%
-4%
8%
14%
5%
19%
-9%
-8%
6%
69%
10%
15%
23%
-7%
19%
32%
-4%
11%
19%
-6%
10%
32%
-3%
6%
39%
-4%
10%
117%
-12%
-5%
-24%
14%
3%
-6%
3%
18%
33%
-13%
4%
89%
-8%
19%
3%
-8%
12%
63%
-2%
33%
-17%
-3%
26%
20%
1%
47%
-22%
-6%
MALE
POP
6%
7%
2%
5%
7%
5%
6%
8%
7%
7%
4%
11%
4%
8%
2%
5%
5%
3%
3%
3%
2%
-1%
2%
5%
9%
9%
4%
LABOUR FORCE
EMP
UEMP
5%
21%
0%
45%
9%
41%
-4%
67%
3%
45%
10%
26%
0%
0%
0%
19%
11%
26%
4%
15%
6%
-8%
4%
46%
6%
22%
13%
20%
6%
15%
3%
16%
3%
23%
2%
63%
-6%
-26%
5%
-2%
6%
19%
13%
100%
15%
2%
15%
54%
24%
-23%
18%
21%
31%
-22%
FEMALE
NILF
4%
14%
-19%
19%
7%
-11%
20%
28%
-6%
12%
0%
25%
-9%
-2%
-7%
6%
3%
-1%
35%
-3%
-8%
-27%
-17%
-17%
-7%
-3%
-16%
POP
5%
6%
7%
3%
6%
6%
1%
7%
8%
7%
3%
9%
4%
5%
2%
3%
1%
1%
1%
2%
1%
1%
4%
7%
7%
8%
8%
LABOUR FORCE
EMP
UEMP
20%
45%
13%
53%
20%
53%
-3%
-6%
26%
96%
27%
79%
20%
57%
1%
48%
46%
53%
22%
9%
69%
-8%
11%
114%
41%
25%
38%
84%
26%
33%
46%
74%
11%
100%
28%
225%
-5%
-19%
-1%
-14%
54%
100%
-12%
50%
28%
19%
2%
150%
79%
44%
98%
0%
338%
-20%
NILF
-1%
3%
-1%
7%
-6%
-8%
-5%
8%
-3%
3%
-10%
3%
-7%
-6%
-5%
-6%
-8%
-18%
6%
7%
-15%
7%
-3%
6%
-2%
3%
-1%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
83
Table A1.13 Growth in population by activity, 2007 to 2010, Age 15-24
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
1%
1%
-5%
2%
4%
-1%
2%
5%
1%
0%
-3%
9%
-2%
2%
-4%
0%
-1%
-5%
-1%
-3%
1%
-2%
-4%
5%
4%
7%
5%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
-1%
10%
1%
-9%
32%
2%
-4%
18%
-8%
-14%
0%
10%
-11%
49%
6%
1%
35%
-7%
-19%
12%
12%
-18%
-3%
20%
16%
20%
-8%
-9%
-14%
5%
10%
-23%
-7%
-15%
57%
21%
15%
3%
-11%
21%
18%
-5%
0%
-20%
-3%
-7%
-6%
3%
-20%
13%
7%
7%
60%
-16%
-10%
-41%
8%
-20%
-26%
9%
13%
50%
-5%
12%
83%
-9%
38%
-8%
-12%
14%
65%
-1%
12%
-23%
4%
57%
4%
-1%
102%
-10%
-7%
MALE
POP
1%
0%
-11%
6%
6%
-3%
9%
7%
-1%
0%
-5%
12%
-4%
5%
-6%
2%
3%
-4%
0%
-4%
1%
-6%
-7%
2%
5%
7%
0%
LABOUR FORCE
EMP
UEMP
-3%
6%
-13%
24%
4%
63%
-6%
17%
-18%
38%
-5%
22%
-23%
-5%
-16%
-15%
13%
3%
-15%
-13%
-3%
-11%
-21%
57%
19%
8%
16%
8%
-4%
-21%
-10%
-10%
-17%
0%
-4%
0%
-14%
-45%
-14%
-14%
0%
40%
34%
100%
24%
-6%
23%
59%
10%
-27%
42%
9%
68%
-6%
FEMALE
NILF
-37%
-53%
-46%
-6%
-42%
-50%
-15%
-34%
-46%
-33%
-56%
-21%
-55%
-27%
-41%
-22%
-29%
-45%
-27%
-34%
-13%
-2%
-36%
-18%
-45%
-25%
-25%
POP
1%
1%
3%
-2%
1%
1%
-5%
3%
3%
0%
-2%
5%
0%
0%
-2%
-2%
-6%
-5%
-3%
-1%
0%
1%
-1%
9%
3%
8%
11%
LABOUR FORCE
EMP
UEMP
4%
19%
-1%
45%
-16%
-22%
-24%
-17%
6%
57%
10%
67%
-12%
100%
-20%
25%
26%
44%
9%
-19%
52%
-31%
-7%
56%
7%
-3%
33%
33%
12%
0%
0%
0%
-24%
50%
24%
400%
-2%
-38%
-26%
-46%
38%
100%
-24%
100%
69%
0%
-10%
133%
19%
0%
125%
-33%
360%
-33%
NILF
-1%
-2%
15%
8%
-7%
-8%
-6%
11%
-6%
1%
-9%
6%
-2%
-8%
-4%
-2%
-3%
-28%
0%
16%
-8%
7%
-8%
9%
1%
2%
2%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
84
Table A1.14 Growth in population by activity, 2007 to 2010, Age 25 to 54
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
5%
7%
5%
2%
5%
5%
1%
6%
7%
7%
4%
8%
4%
6%
2%
3%
1%
1%
0%
1%
0%
0%
4%
5%
9%
8%
5%
MALE AND FEMALE
LABOUR
FORCE
EMP
UEMP NILF
10%
37%
-4%
5%
53%
5%
12%
64%
-13%
-2%
57%
3%
11%
68%
-11%
15%
41%
-15%
7%
9%
-7%
4%
47%
6%
15%
46%
-6%
10%
30%
2%
18%
-2%
-11%
9%
69%
1%
13%
36%
-12%
17%
43%
-8%
13%
50%
-13%
11%
53%
-9%
9%
65%
-16%
13%
167%
-25%
-3%
-12%
9%
3%
19%
-5%
18%
8%
-24%
6%
83%
-14%
15%
8%
-9%
14%
61%
-8%
40%
-14%
-11%
18%
35%
1%
35%
-29%
-7%
MALE
POP
6%
7%
4%
2%
5%
5%
2%
7%
8%
7%
5%
9%
4%
7%
3%
4%
2%
2%
1%
2%
1%
1%
4%
6%
9%
9%
5%
LABOUR FORCE
EMP
UEMP
5%
27%
2%
50%
7%
29%
-3%
110%
5%
42%
9%
28%
1%
-3%
3%
34%
7%
39%
6%
25%
5%
-3%
8%
34%
3%
28%
10%
25%
7%
53%
2%
26%
3%
43%
5%
100%
-3%
-13%
4%
11%
7%
10%
12%
91%
11%
12%
16%
57%
29%
-23%
12%
29%
23%
-28%
FEMALE
NILF
-86%
-85%
-89%
-88%
-87%
-87%
-86%
-88%
-87%
-86%
-88%
-86%
-89%
-79%
-87%
-86%
-87%
-89%
-87%
-91%
-93%
-81%
-86%
-82%
-88%
-71%
-80%
POP
5%
7%
5%
2%
5%
5%
1%
6%
7%
7%
3%
8%
4%
5%
1%
2%
0%
0%
0%
0%
-1%
-1%
3%
4%
8%
8%
5%
LABOUR FORCE
EMP
UEMP
24%
67%
18%
60%
29%
100%
2%
10%
29%
138%
30%
86%
27%
30%
7%
79%
48%
65%
23%
39%
68%
40%
13%
146%
49%
49%
37%
109%
32%
60%
59%
214%
22%
100%
35%
250%
-2%
-6%
2%
33%
53%
100%
-5%
0%
23%
10%
5%
250%
122%
167%
85%
50%
300%
100%
NILF
-4%
2%
-11%
1%
-11%
-13%
-9%
4%
-6%
0%
-14%
-1%
-13%
-8%
-10%
-12%
-15%
-24%
2%
-3%
-20%
2%
-4%
2%
-5%
3%
-4%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
85
Table A1.15 Growth in population by activity, 2007 to 2010, Age 55 to 64
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
POP
19%
17%
24%
16%
18%
20%
18%
19%
21%
22%
18%
25%
22%
23%
15%
16%
20%
17%
15%
20%
12%
10%
19%
18%
16%
14%
15%
MALE AND FEMALE
LABOUR FORCE
EMP
UEMP
NILF
24%
72%
16%
14%
122%
16%
42%
50%
15%
-7%
0%
30%
33%
150%
13%
51%
400%
7%
38%
9%
5%
0%
23%
73%
25%
8%
33%
133%
18%
59%
33%
2%
29%
200%
20%
41%
29%
16%
46%
0%
14%
16%
100%
13%
55%
300%
4%
23%
0%
18%
-3%
100%
31%
-12%
-67%
42%
29%
-100%
12%
25%
0%
-25%
41%
29%
100%
13%
-14%
100%
40%
30%
0%
13%
26%
-67%
13%
36%
0%
9%
POP
19%
17%
24%
16%
18%
22%
19%
20%
21%
22%
19%
25%
24%
24%
17%
19%
20%
18%
16%
21%
13%
11%
21%
17%
18%
18%
18%
MALE
LABOUR FORCE
EMP
UEMP
21%
74%
10%
138%
37%
50%
-10%
0%
26%
167%
51%
33%
38%
4%
63%
33%
24%
200%
42%
-33%
19%
150%
24%
50%
30%
33%
19%
-50%
46%
200%
33%
0%
-10%
-8%
-67%
42%
-50%
6%
0%
-17%
40%
-33%
-22%
-100%
17%
50%
19%
0%
31%
-100%
NILF
-30%
-14%
-35%
-35%
-25%
-35%
-35%
-8%
-29%
-17%
-46%
-34%
-25%
-42%
-33%
-25%
-46%
-40%
-40%
-55%
-48%
-49%
-42%
-48%
-28%
-39%
-41%
POP
18%
17%
23%
15%
19%
19%
17%
18%
21%
21%
17%
24%
21%
22%
13%
14%
19%
16%
15%
19%
11%
8%
18%
20%
14%
10%
12%
FEMALE
LABOUR FORCE
EMP
UEMP
31%
75%
45%
0%
44%
0%
-100%
80%
-100%
60%
42%
0%
-100%
100%
86%
-100%
138%
-100%
43%
100%
0%
57% 9% 86%
-100%
11%
8%
0%
-20%
13%
100%
-40%
14%
20%
0%
100%
-100%
67%
#DIV/0!
-100%
NILF
15%
16%
16%
22%
16%
12%
12%
21%
14%
19%
6%
17%
12%
13%
14%
9%
21%
20%
35%
24%
-11%
33%
15%
20%
10%
7%
10%
Source: Own calculations based upon LFS, TÜİK/TURKSTAT
Abbreviations: POP – Population; EMP – Employed; UEMP – Unemployed; NILF – Not in labour force
86
Table A1.16 Employment rates by age group 2010, Male and Female
2010
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
MALE
15-24
25-54
37%
46%
37%
35%
44%
45%
39%
46%
36%
36%
47%
46%
35%
30%
37%
32%
33%
43%
39%
30%
28%
23%
23%
27%
40%
24%
18%
55-64
78%
81%
82%
86%
78%
77%
81%
84%
80%
81%
83%
82%
79%
70%
77%
79%
78%
82%
84%
84%
83%
67%
73%
65%
67%
63%
63%
39%
25%
44%
51%
30%
39%
41%
35%
32%
27%
45%
49%
38%
46%
41%
33%
48%
57%
59%
57%
52%
69%
42%
73%
42%
40%
44%
TOTAL
63%
67%
67%
69%
64%
65%
65%
69%
64%
64%
69%
70%
63%
56%
62%
61%
63%
68%
69%
67%
62%
51%
55%
51%
57%
47%
44%
FEMALE
15-24
19%
23%
29%
29%
22%
26%
21%
25%
18%
11%
14%
29%
19%
14%
13%
13%
30%
29%
27%
31%
16%
20%
9%
10%
12%
5%
3%
25-54
55-64
25%
20%
32%
37%
28%
33%
25%
28%
20%
25%
20%
39%
22%
23%
24%
15%
33%
34%
39%
55%
26%
42%
24%
20%
9%
6%
3%
14%
3%
15%
24%
6%
14%
15%
9%
9%
4%
10%
23%
10%
17%
19%
8%
42%
33%
37%
51%
20%
34%
26%
14%
7%
5%
0%
TOTAL
22%
19%
29%
33%
24%
29%
23%
25%
18%
19%
17%
35%
20%
20%
20%
14%
34%
33%
36%
48%
22%
33%
20%
16%
10%
5%
3%
Source: Own calculation based upon LFS series, TÜİK/TURKSTAT
87
Table A1.17 Growth in employment rates by age group 2007 to 2010, Male and Female (percentage points)
CODE
TR
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
GEO
Türkiye
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
MALE
15-24
-1.6%
-5.9%
6.5%
-4.0%
-10.0%
-1.1%
-11.2%
-9.9%
4.9%
-5.2%
0.7%
-13.7%
8.4%
3.2%
0.6%
-3.7%
-6.7%
0.1%
-5.5%
-3.2%
-0.2%
9.8%
7.6%
5.5%
1.8%
7.8%
12.4%
25-54
-0.2%
-4.4%
2.0%
-3.6%
0.2%
3.2%
-0.6%
-2.9%
-0.5%
-1.2%
0.2%
-0.9%
-1.1%
2.0%
2.3%
-1.5%
0.6%
2.2%
-3.7%
1.4%
4.8%
7.4%
5.1%
6.5%
12.3%
1.8%
10.5%
55-64
0.6%
-1.4%
4.6%
-11.6%
2.0%
9.5%
6.6%
-4.4%
11.0%
0.4%
9.1%
-2.7%
0.2%
2.1%
0.8%
7.7%
5.4%
-13.8%
-11.9%
9.9%
-3.2%
-17.3%
6.8%
-24.2%
-0.4%
0.5%
4.8%
TOTAL
-0.5%
-4.4%
4.0%
-5.6%
-2.5%
2.9%
-3.5%
-5.2%
2.2%
-2.0%
1.6%
-4.8%
1.5%
2.3%
2.0%
-1.3%
-0.9%
-0.9%
-5.5%
1.8%
2.3%
7.0%
6.8%
4.7%
8.2%
3.9%
11.4%
FEMALE
15-24
0.6%
-0.4%
-5.4%
-6.5%
1.2%
2.2%
-1.5%
-5.8%
4.0%
1.0%
7.4%
-3.3%
1.3%
4.8%
1.8%
0.2%
-5.6%
9.0%
0.3%
-7.9%
5.9%
-4.9%
6.2%
-1.8%
1.9%
5.7%
8.6%
25-54
4.7%
2.1%
7.2%
-0.1%
6.4%
7.9%
6.7%
0.3%
7.7%
3.7%
12.5%
1.8%
9.5%
7.1%
7.4%
8.3%
7.5%
11.9%
-0.8%
0.8%
14.1%
-1.8%
4.8%
0.2%
9.6%
4.0%
7.9%
55-64
1.6%
0.7%
2.5%
-3.1%
3.2%
4.7%
3.4%
-1.4%
5.8%
2.3%
10.2%
3.5%
6.8%
4.8%
-0.7%
5.1%
-3.1%
-2.3%
-11.2%
-2.2%
15.9%
-15.0%
-0.9%
0.0%
4.9%
2.5%
4.8%
TOTAL
3.2%
1.2%
3.6%
-2.0%
4.6%
6.0%
4.2%
-1.5%
6.5%
2.8%
10.8%
0.7%
7.0%
6.2%
4.9%
5.7%
3.4%
8.7%
-2.0%
-1.6%
11.8%
-4.3%
4.7%
-0.7%
6.7%
4.5%
7.9%
Source: Own calculation based upon LFS series, TÜİK/TURKSTAT
88
Table A1.18 Share of Population aged 15 and over by educational attainment, 2010, Male and Female
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
TURKEY
Illiterate
3.7%
4.4%
5.7%
4.3%
5.9%
7.4%
4.3%
5.0%
3.5%
6.4%
3.6%
7.2%
9.1%
8.1%
7.6%
8.5%
10.9%
9.0%
9.9%
10.3%
13.7%
11.1%
16.2%
10.3%
16.7%
17.1%
7.0%
Literate but
no school
completed
4.3%
4.3%
5.1%
4.1%
4.5%
5.2%
3.8%
4.3%
2.9%
3.9%
5.2%
5.8%
6.3%
4.9%
4.9%
5.6%
7.1%
6.3%
6.2%
7.6%
13.6%
7.2%
16.2%
7.9%
16.4%
13.6%
5.9%
Junior high
school or
vocational
school at
junior high
school level
graduate
6.7%
6.5%
5.3%
6.4%
5.1%
4.9%
7.2%
6.3%
7.2%
4.8%
5.5%
6.0%
5.4%
6.1%
5.5%
5.3%
4.9%
4.7%
5.7%
4.9%
3.4%
5.7%
2.8%
4.8%
2.8%
3.0%
5.7%
High school
or
vocational
school at
high school
level
graduate
23.4%
22.6%
19.4%
23.2%
18.9%
17.9%
23.9%
22.7%
27.4%
17.2%
21.9%
21.8%
18.6%
19.3%
20.7%
20.0%
16.3%
17.5%
21.0%
20.2%
13.5%
21.3%
13.7%
16.0%
13.2%
15.0%
20.8%
Higher
education
graduate
and more
11.8%
8.5%
8.9%
12.1%
9.1%
6.8%
9.7%
8.8%
17.0%
7.8%
10.2%
8.8%
6.5%
7.1%
7.8%
7.4%
6.5%
7.2%
7.8%
7.5%
4.2%
7.8%
4.0%
5.6%
4.3%
4.2%
9.2%
Unknown
6.3%
3.4%
3.7%
4.7%
4.4%
2.5%
4.0%
4.4%
4.7%
3.0%
4.8%
4.5%
4.7%
3.7%
6.2%
3.0%
3.0%
4.4%
4.6%
5.6%
10.7%
5.3%
6.9%
5.4%
7.4%
6.0%
5.0%
TOTAL
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
Source: Own calculation based upon statistics by TÜİK/TURKSTAT
89
Table A1.19 Share of Population aged 15 and over by educational attainment, 2010, Male
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
TURKEY
Illiterate
1.2%
1.9%
2.6%
1.6%
2.0%
2.7%
1.5%
1.7%
1.0%
2.0%
1.2%
2.7%
3.5%
2.4%
2.7%
2.9%
4.8%
4.0%
3.3%
3.7%
5.2%
4.4%
6.5%
3.9%
6.5%
6.8%
2.6%
Literate but
no school
completed
3.1%
3.4%
4.3%
3.0%
3.2%
4.0%
2.6%
2.9%
1.8%
2.8%
3.3%
4.3%
4.6%
3.4%
3.7%
3.7%
5.6%
5.0%
4.5%
5.8%
11.2%
4.9%
12.2%
5.8%
12.7%
9.8%
4.3%
Junior high
school or
vocational
school at
junior high
school level
graduate
7.9%
7.7%
6.7%
7.4%
6.4%
6.6%
8.7%
8.1%
8.6%
6.7%
6.8%
7.3%
6.8%
8.1%
7.1%
7.2%
7.2%
6.2%
7.6%
6.7%
4.5%
7.7%
4.0%
6.5%
4.1%
4.2%
7.1%
High school
or
vocational
school at
high school
level
graduate
25.1%
25.6%
22.6%
25.0%
20.6%
21.9%
27.8%
27.0%
29.9%
21.2%
24.3%
24.2%
22.0%
23.5%
25.0%
24.8%
20.7%
21.0%
25.2%
26.0%
17.6%
26.5%
19.4%
19.6%
17.9%
21.5%
24.0%
Higher
education
graduate
13.0%
9.7%
10.7%
13.3%
10.5%
8.5%
11.6%
10.7%
19.3%
10.5%
11.8%
10.4%
8.2%
9.2%
9.9%
9.1%
8.5%
9.2%
9.9%
9.8%
5.2%
10.2%
5.6%
7.3%
5.9%
6.2%
10.9%
Unknown
6.9%
3.9%
4.2%
5.3%
5.0%
3.0%
4.2%
4.5%
5.0%
3.4%
5.5%
5.0%
5.3%
4.0%
6.1%
3.1%
3.3%
4.6%
4.6%
5.6%
10.9%
5.4%
6.4%
5.7%
6.6%
5.6%
5.3%
TOTAL
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
Source: Own calculation based upon statistics by TÜİK/TURKSTAT
90
Table A1.20 Share of Population aged 15 and over by educational attainment, 2010, Female
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
Istanbul
Tekirdag
Balikesir
Izmir
Aydin
Manisa
Bursa
Kocaeli
Ankara
Konya
Antalya
Adana
Hatay
Kirikkale
Kayseri
Zonguldak
Kastamonu
Samsun
Trabzon
Erzurum
Agri
Malatya
Van
Gaziantep
Sanliurfa
Mardin
TURKEY
Illiterate
6.2%
7.0%
8.9%
7.0%
9.7%
12.3%
7.1%
8.2%
6.0%
10.6%
6.1%
11.6%
14.7%
13.6%
12.6%
14.1%
16.8%
13.9%
16.3%
17.1%
22.6%
17.7%
26.2%
16.7%
26.7%
27.3%
11.4%
Literate but
no school
completed
5.5%
5.3%
5.9%
5.3%
5.8%
6.4%
5.0%
5.8%
4.0%
5.0%
7.2%
7.3%
8.1%
6.4%
6.2%
7.5%
8.6%
7.5%
7.8%
9.5%
16.0%
9.4%
20.3%
9.9%
20.1%
17.3%
7.4%
Junior high
school or
vocational
school at
junior high
school level
graduate
5.5%
5.2%
4.0%
5.5%
3.8%
3.3%
5.7%
4.6%
5.9%
2.9%
4.2%
4.8%
3.9%
4.3%
3.9%
3.5%
2.8%
3.2%
3.9%
3.0%
2.1%
3.7%
1.6%
3.1%
1.6%
1.8%
4.3%
High school
or
vocational
school at
high school
level
graduate
21.7%
19.6%
16.1%
21.4%
17.2%
13.8%
19.9%
18.5%
25.0%
13.4%
19.4%
19.4%
15.1%
15.2%
16.4%
15.3%
12.1%
14.2%
16.9%
14.3%
9.2%
16.3%
7.9%
12.4%
8.7%
8.6%
17.6%
Higher
education
graduate
10.6%
7.2%
7.0%
10.9%
7.7%
5.0%
7.9%
6.8%
14.8%
5.4%
8.5%
7.1%
4.7%
5.0%
5.7%
5.7%
4.6%
5.4%
5.7%
5.1%
3.0%
5.4%
2.4%
3.9%
2.8%
2.3%
7.5%
Unknown
5.7%
2.9%
3.3%
4.1%
3.8%
2.0%
3.8%
4.4%
4.5%
2.7%
4.1%
4.0%
4.1%
3.5%
6.2%
3.0%
2.7%
4.2%
4.6%
5.6%
10.5%
5.3%
7.5%
5.1%
8.1%
6.5%
4.7%
TOTAL
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
100.0%
Source: Own calculation based upon statistics by TÜİK/TURKSTAT
91
Annex 2. Major developments since 2007
1. Major Developments since 2007 at National Level
The HRD-OP (2007-2009) aims to address the main challenges Turkey faces in the fields of
employment, education and training, and social inclusion. The Programme has been prepared in
compliance with Turkey's 9th National Development Plan (NDP), Turkey's Multi-annual Indicative
Planning Document (MIPD) and the Strategic Coherence Framework (SCF). The priorities and
measures set out in HRD OP reflect the key problems and challenges identified in the NDP, the MIPD,
the SCF, and the joint strategy papers. The SCF, the JIM and the JAP have been jointly elaborated by
Turkey and the European Commission. The major developments since HRD-OP (2007-2009)
produced are summarised below.
Implementation of Partnership Approach
We observed that during the implementation of the HRD-OP strong collaborations have established
with relevant public institutions, non-governmental organisations, social partners, and universities
during different phases of implementations of HRD-OP, although some members of organisations did
not fount the collaboration sufficient. This collaborative approach helped to improve HRD-OP
mechanism, enabled stakeholders share their experience and knowledge, built up opportunities for
more creative ways to address problems and allowed a more effective management.
Sectoral Monitoring Committee
The Sectoral Monitoring Committee, a permanently acting body of HRD-OP established under the
provisions of Article 36 of the Framework Agreement between the government of the Republic of
Turkey and the European Commission, had its first meeting on 13 December 2007 and hence after it
held meetings twice a year at the initiative of the Operating Structure or the Commission. These
meetings contributed greatly to the development and implementation of HRD-OP. The dates of the
meetings are presented in Table 3.
Table 3: The date of Sectoral Committee Meetings
Meetings
Date
Location
1st Sectoral Committee Meeting
13.12.2007
Ankara
2nd
Sectoral Committee Meeting
17.06.2008
Ankara
3rd
Sectoral Committee Meeting
13.11.2008
Ankara
4th Sectoral Committee Meeting
29.07.2009
Ankara
5th Sectoral Committee Meeting
23.11.2009
Ankara
6th
Sectoral Committee Meeting
24.06.2010
Van
7th
Sectoral Committee Meeting
30.11.2010
Hatay
8th
Sectoral Committee Meeting
07.07.2011
Trabzon
Services Provided by Different Institutions Related to HRD-OP Target Groups
Several activities in different institutions have been carried out in order to improve human resources of
the particular groups related to the priority axis of HRD-OP. The Ministry of Labour and Social
Security has been working on the preparation of a national employment strategy. One of the main
aims of this strategy is to determine measures that will increase employment of disadvantaged
developed during a series of workshops with the participation of over 40 relevant bodies and
stakeholder Ministry has also prepared an action plan to strengthen relationships between vocational
education and employment of these groups prioritised in HRD-OP.
92
IŞKUR, with UMEM Project has determined the demand of the workforce market through need
analyses in each province. The vocational and technical teachers of 100 vocational high schools have
been trained. Students and non-students could both attend these trainings.
The General Directorate of Technical Education for Boys and the General Directorate of Technical
Education for Girls provided various services or contributed to different projects at institutional level to
improve education and employment of young people. During 2009, in accordance with the request of
the Ministry of Labour and Social Security, the General Directorate of Technical Education for Boys
worked with the Ministry of National Education and Ministry of Industry and Commerce in order to
discuss the measures needed to overcome the supply and demand discrepancy between technical
education and workforce demand. TOBB also contributed to this study. The General Directorate of
Technical Education for Boys also contributed to the Beceri’10 Project of Specialized Career Planning
Centers (UMEM) which is primarily focused on training and employment. The Turkish Union of
Chambers and Stock Markets (TOBB), the Ministry of Labour and Social Security, the Ministry of
National Education and the TOBB University of Economics and Technology (TOBB-ETÜ) have signed
a protocol in order to implement the pilot project in 19 provinces. The executive board of the project
consists of Deputy Secretary of Ministry of Labour and Social Security, General Manager of İŞKUR,
Director of General Directorate of Technical Education for Boys, General Secretary of TOBB and
TOBB-ETÜ President. On-the-job trainings for teachers were opened under UMEM. 2,000 teachers
already attended these courses and 4,000 more will be trained.
The Ministry of Development (formerly SPO) has been implementing SODES (Social Support
Program) in Eastern and Southeastern Anatolia. This program supports vocational trainings, sports
activities and various courses for youth, fragile, disadvantaged groups. This is one of the most
comprehensive and successful projects on social inclusion in Turkey. It encourages the development
of projects created by local initiatives and NGO partnerships.
Collaborations amongst Related Institutions
Several institutions collaborate at certain level with each other in order to improve employment,
education, and social inclusion. The Ministry of Labour and Social Security implements their action
plan with the coordination of the Ministry of Industry and Commerce and the Ministry of National
Education. They hold annual meetings at the end of each year with the participation of the three
ministers in order to secure political support at the top management level.
İŞKUR have collaborations with various public and private bodies such as the Ministry of National
Education, KOSGEB, Turkcell, Turk Telecom, General Directorate of Social Support and Solidarity
and Ministry of Forests and Environment.
The General Directorate of Technical Education for Boys has been collaborating with different
institutions for different projects setting. It has close collaboration with the Ministry of Labour and
Social Security, Ministry of National Education, Turkish Union of Chambers and Stock Markets
(TOBB), and TOBB University of Economics and Technology (TOBB-ETÜ) for UMEM Project, and
with the Ministry of National Education and Ankara Chamber of Industry (ASO) for OSEP –School
Industry Education Program.
The General Directorate of Commerce and Tourism Education have intensive collaboration with the
Ministry of Culture and Tourism. The Directorate also collaborates with İŞKUR within UMEM project,
and collaborates with TÜRSAB in order to meet the qualified workforce demand of the sector.
The General Directorate of Technical Education for Girls also collaborates with other public, private
and NGO partners in order to strengthen the relationship between vocational education and
employment, to contribute to the development of vocational and technical education, to solve the
problems in education system and to create awareness.
93
The Ministry of Development has partnerships with public institutions, youth and other NGOs. It also
provides guidance to other public institutions.
Evaluation of Institutional Collaborations
Although institutions collaborate with each other, we noticed that this is usually based on projectbased partnership. If they are not conducting the same project, they are not usually having sufficient
information on the services they provide. This has naturally a negative effect on the creation of an
integrated policy with common targets. At the same time it reduces the quality and effectiveness of
measures taken by individual Institutions. We believe that common plans in a strategic framework
would have much enhanced the application of such policies and would have contributed to them
having a lasting effect in the Turkish society rather than “project-duration” effect. Also, such a deeper
and more permanent collaboration would have helped to direct better the efforts spent and balance
the distribution of projects according to the whole spectrum of needs rather than randomly covering
specific areas that happened at a certain time to arise.
Also we noticed that Institutions are not always able to keep consistent, detailed records of their
services and information relevant to beneficiaries and all their stakeholders. There is also lack of a well
working database system that will allow different bodies to access the relevant information. Therefore
there is lack of effective and continuous communication and this result to often having projects that
duplicate previously run activities.
Development in Higher Education
The Turkish Higher Education system has seen a big increase both in numbers of Universities and
consequently in numbers of students. Since 2007, 50 new Universities have been established bringing
their total University number from 115 to 165. From these, 30 are Foundation (private) Universities and
18 are State Universities. This makes for a large and imbalanced increase in favour of the Private
sector: The number of students enrolled at Higher Education establishments has also increased over
30% from 2007 to 2011. Comparing the actual numbers, 1.075.670 more students were registered in
2011 than in 2007 (Özcan, 2011) 26. Most of the State Universities are established in the NUTS-2
region which has a high proportion of people belonging to disadvantaged groups. We believe that this
will increase the opportunities for members of such groups to access higher education.
Removing Coefficient System to Promote of Vocational and Technical Secondary Schools’
Students to Universities
A change, in effect from 2012, will be implemented in order to equalise the opportunities of the
students of Vocational and Technical High Schools to enter Universities. This was to abolish the
coefficient system which was applied to University entrance exam scores and lowered the marks of
the students from such High Schools. This system was in effect since 1998. The chance is expected to
increase the enrolment to Vocational and Technical High Schools. Nevertheless, more measures need
to be taken to also increase the quality of education in these schools.
Vocational Qualification Authority (VQA)
Under the coordination of the Vocational Qualifications Authority (VQA), centres for sectoral
occupational standards development, skill and knowledge testing and certification’ were established
and started to develop occupational standards and infrastructure for knowledge and skills tests in
cooperation with relevant NGOs and private-sector organisations. Turkey needs to continue its efforts
to establish a national qualifications system applying to all types of education, training and
qualifications and linked to the European Qualification Framework.
26
Reference: Özcan, Y. Z. (2011). Challenges to the Turkish Higher Education System. Paper presented in the
22nd International Conference on Higher Education. Bilkent University, Ankara (June 17-19, 2011).
94
During the implementation of grant scheme projects, a variety of vocational training activities were
conducted and some of them are still being conducted. However, at present, there is no
standardization of these trainings. In order to increase the quality of the project activities, the use and
compliance of these standards should be mandatory for the vocational trainings to be provided under
OP projects.
Major HRD-Related Projects Implemented since 2007
The Major HRD-related projects implemented since 2007 are outlined below.
Table 1: Major HRD Related Projects
Project Name
Objectives
Starting Date
Beneficiaries
Promoting Women's
Employment Grant Scheme
To increase women employment, to
help women to enter more jobs and
better jobs, and to reduce barriers
preventing women employment
30 July 2010
İŞKUR
Promoting Youth Employment
Grant Scheme
To increase youth employment, to
organize entrepreneurship training at
local level and deliver counseling and
guidance services for establishing their
own jobs, to increase the opportunities
of internship in order students to transit
from schools to jobs easily, to increase
vocational qualifications of youth
people, and increase the number of
people
who
benefitted
from
apprenticeship programme.
23 Nov. 2010
İŞKUR
Increasing School Enrolment
Rates Especially for Girls
Grant Scheme
To enhance investment in human
capital by increasing the quality of
education, improving the linkage
between education and the labour
market, and raising enrolment rates at
all levels of education, especially for
girls within the scope of positive
discrimination for women in the EU
adaptation process.
Jan. 2011
MoNE
Promoting of Life Long
Learning Grant Scheme
To transfer dynamic labour force,
employment, continuous learning and
culture of learning and qualitative
education access compliant with
different
age
groups,
improved
technology and the demands of labour
force into a lifelong learning strategy for
adaptation in global changes of today's
world in which social and economic
power depends on knowledge.
15 May 2010
MoNE
Promoting Registered
Employment through
Innovative Measures Grant
Scheme
To build institutional capacity of the
Social
Security
Institution,
other
relevant institutions, social partners and
relevant NGOs in order to promote
registered employment and to support
those mentioned authorities to prepare
and implement their strategies in
related areas.
15 July 2010
SSI
95
Major Outcomes and Evaluation of the HRD-OP Projects
An immense number of institutions have benefitted from grant projects of HRD-OP. Among them trade
associations, chambers, federations-confederations, foundations, unions, NGO’s, municipalities,
universities, vocational secondary schools, public training centres, primary schools, industrial zones
and farmer groups.
The data indicate that 44.792 people benefitted or will benefit from the various training activities. The
number of trainees is 13.002 in “Promoting Women's Employment”, 11.490 in “Promoting Youth
Employment”, 8238 in “Increasing School Enrolment Rates Especially for Girls”, 7811 in “Promoting
Registered Employment through Innovative Measures”, and 4251 in “Promoting of Life Long
Learning”.
While the data were collected from MIS, training activities of the some projects were still going on. At
the data collection time, the percentage of trainees who completed their training was 41% in
“Promoting Registered Employment through Innovative Measures”, 36% in “Promoting Women's
Employment”, 28% in “Promoting of Life Long Learning”, 26% in “Promoting Youth Employment”, and
10% in “Increasing School Enrolment Rates Especially For Girls”. The ratio of unsuccessful trainees
varies from 2.2% (Promoting Registered Employment through Innovative Measures) to 4.9%
(Promoting Youth Employment). The unsuccessful students failed or simply left the training.
The trainings had all very high rates of success. The percentages of successful trainees varied from
98% (Promoting Registered Employment through Innovative Measures) to 88% (Increasing School
Enrolment Rates Especially for Girls). About half of the unsuccessful trainees failed because of
absentees.
Most of the training topics in these projects are categorised as “general/occupational skills”,” child
and/or elderly care training”, “entrepreneurship training” (Promoting Women's Employment),
“informative meetings or training for families or parents on promotion of girls’ education”, “informative
meetings or training for social partners on promotion of girls’ education”, “basic skills (reading, writing
etc.)” and “general occupational skills” (Increasing School Enrolment Rates Especially For Girls),
“general occupational skills”, “employment guarantee vocational training”, and “entrepreneurship
training” (Promoting Youth Employment), “general occupational skills”, “employment guarantee
vocational training”, “general skill training”, and “basic skills” (Promoting of Life Long Learning), and
above 90% of the trainees trained on general occupational skills, “informatory meeting or training for
social partners to foster registered employment” and “general skill training” (Promoting Registered
Employment through Innovative Measures).
The gender distribution of the trainees naturally varied according to the projects. For example, the
percentage of females is 100% in “Promoting Women's Employment”, it is 86% in “Increasing School
Enrolment Rates Especially For Girls” and slightly less than 50% in “Promoting of Life Long Learning”.
However, the percentage of females is 38.9% in “Promoting Youth Employment” and 37% in
“Promoting Registered Employment through Innovative Measures”.
Most of the trainees are either primary or secondary school graduates. The percentages of trainees
graduated from primary and secondary schools vary from 65% (Promoting Youth Employment and
Promoting of Life Long Learning) to 85% (Promoting Women's Employment).
The age of the trainees vary from one grant project to another. For example, the youth and adult were
nearly evenly represented both in “Promoting Women's Employment” and “Promoting Registered
Employment through Innovative Measures”. However, only 1/3 of the trainees were youth in
“Promoting of Life Long Learning”, nearly 80% of them were youth in “Increasing School Enrolment
Rates Especially for Girls” and about 99% in “Promoting Youth Employment”.
96
In the case of social exclusion groups, the percentages varied from 1% to 9%. In the “Increasing
School Enrolment Rates Especially for Girls” about 9% of the participants were from disadvantaged
groups with most of them originating from two groups: 1) “Poor or under risk of poorness” and 2)
“Other people in need of special attention (substance addiction, victimised women in family,
family/parents of working children)” were the majority.
In the case of “Promoting Youth Employment”, the percentage of the trainees whose handicapped
level above 25%, terror victims, Romany people who needs special attention, Ex convicted, Poor or
under risk of poorness, including one who live in shanty towns (Gecekondu areas), and other people
in need of special attention (substance addiction, victimised women in family, family/parents of working
children) was about 5%. When Poor or under risk of poorness were eliminated the percentage of the
remaining groups made for less than 1% of the participants. In the remaining projects the percentage
of disadvantaged groups was less than 1%.
Since HRD-OP is expected to develop final beneficiaries, the data given above revealed that
participation rates in employment, particularly for women and young people have been increased to a
certain level, after the training activities conducted through project activities. However, since the
project activities are still going on, the rate of increase in enrolment and decrease of dropouts,
particularly of girls in secondary/VET education will give a clearer picture later, after completions of the
projects.
Nevertheless, the data collected from MIS indicates that the participations of handicapped people,
terror victims, Romany people who needs special attention, ex convicted, poor or under risk of
poorness and other people in need of special attention (substance addiction, victimized women in
family, family/parents of working children were limited.
Other HRD Related Projects
In addition to the above projects, other Projects played a positive role on human resources
development in less developed regions. These are outlined below.
Table 2: Other HRD-related projects implemented since 2007
Project Name
Objectives
Duration
Partners
GAP-2 Project
To develop human resources and reduce
unemployment rate in the provinces of GAP region
(Gaziantep, Adıyaman, Diyarbakır, Şırnak, Siirt,
Batman, Şanlıurfa, Kilis and Mardin).
To develop and implement labour market
intervention instruments for decreasing long term
unemployment.
2008-2012
İŞKUR
2008 –2011
To promote employability of fragile groups of
youth, youth women and youth immigrant families.
İŞKUR is expected to develop a national youth
employment action plan for Turkey.
Formulating proposals for Strategic Plan and
Action Plan for Increasing Women Employment
and developing of “Handbook of Gender Equality”.
2008-2001
İŞKUR,
Ministry of
Social
Services and
Employment
(Netherland)
İŞKUR,
UNDP, ILO,
FAO, IOM
2009-2010
İŞKUR and
ILO
To strengthen gender equality within labour
market and raise awareness on it.
2009-2010
İŞKUR and
British Council
To improve labour force for the future and
contribute to the development of SMEs in the
region and their competition by increasing the
quality of vocational education in 8 provinces of
2008-2010
MoNE
Improving Labour Market
Intervention to the Benefit
of Employability of
Long Term Unemployed
Development with Human
Caring Job: Youth
Employment Programme in
Antalya
Gender Equality by
Providing Humanistic
Employment Opportunities
for Women
Strategy Development
Project for Promoting
Women Employment
Support to Human
Resources Development
through VET
97
Project Name
Strengthening the
Administrative Capacity of
MoNE (MEBGEP)
Active Employment
Measures and Support to
Turkish Employment
Organization at Local Level
Support Women
Entrepreneurship Through
Training Centres and
Relays in Turkey
Objectives
Eastern and South-eastern Anatolia Region.
To evolve an action plan to facilitate improvement
of the capacity of MoNE in the fields of
administration, management and organization,
handling financial resources and
monitoring/evaluating in order to make the system
more effective and productive in the process of
restructuring.
To assist İŞKUR to increase its capacity to design
and implement measures in order to boost
employment and employability of specific target
groups of women and young people.
To address the low participation of women in
Turkey’s economic life, to promote
entrepreneurship among them and enhance their
role in society. The Project aimed to establish 9
training centres and 16 training relays in different
parts of Turkey as well as train and provide
consultancy to more than 5000 women.
Duration
Partners
2008-2009
MoNE
2007-2009
İŞKUR
2007-2009
TESK
KOSGEB
Yearly Major Developments in the Implementations of the HRD-OP
Major developments in the implementation of the HRD-OP in the years of 2007, 2008, 2009, and 2010
are summarised separately below.
Major Developments in the Implementation of the HRD-OP in 2007

The MoLSS, as designated Operating Structure (OS), established an HRD Technical
Committee for the preparation of the HRD OP. On 1 May 2007, the MoLSS provided the first
draft HRD OP to the European Commission. Between May and October 2007, discussions
took place between the MoLSS and the Commission in view of finalizing the HRD OP. The
drafts were submitted to the Commission for comments.

The following relevant institutions and social partners were consulted throughout HRD-OP
preparations: 1) Ministry of Finance, 2) Ministry of National Education, 3) Ministry of Health, 4)
Ministry of Transport, 5) Ministry of Food, Agriculture and Livestock, 6) Ministry of Science,
Industry and Technology, 7) Ministry of Environment and Urban Affairs, 8) State Planning
Organization, 9) Under secretariat of Treasury, 10) Ministry for European Union Affairs, 11)
DG on Social Services and Child Protection, 12) DG on Social Assistance and Solidarity, 13)
DG on Foundations, 14) DG on Women Status, 15) DG on Family and Social Research, 16)
Council of Higher Education, 17) Housing Development Administration, 18) Administration for
the Disabled People, 19) Small and Medium Sized Business Development Organization
(KOSGEB), 20) National Productivity Centre, 21) Confederation of Turkish Employer
Organizations (TISK), 22) Confederation of Turkish Employees (TURK-İŞ), 23) HAK-İŞ
Confederation (HAK-İŞ), 24) DISK Confederation (DİSK), 25) KAMUSEN Confederation
(KAMUSEN), 26) KESK Confederation (KESK), 27) MEMURSEN Confederation
(MEMURSEN), 28) BASK Confederation (BASK), 29) The Union of Chambers and
Commodity Exchanges in Turkey (TOBB), 30) Confederation of Artisans and Craftsmen
(TESK), 31) 15) TURKSTAT.

In order to ensure ownership of the HRD OP, the 43 Provincial Governorships in the 12
eligible NUTS II regions were consulted.

An ex-ante evaluation on the HRD OP was carried out under the responsibility of the MoLSS.
The results of the ex-ante evaluation were taken into account in the drafting process of the
HRD OP.
98

The Operation Identification Sheet (OIS) was elaborated.

The following institutions were designated for the implementations of the 2007 allocation of the
HRD OP: 1) Measure 1.1: Public Employment Agency, 2) Measure 1.3: Social Security
Institution – Guidance and Inspection Board, 3) Measure 3.1: Ministry of National Education,
4) Measure 4.2: Social Security Institution – DG on Non Contributory Payments.

A Technical Committee has been established in collaboration with the Ministry of Science,
Industry and Technology to ensure regular dialogue and exchange of information on the
interventions of the HRD-OP and the RC-OP. This Committee has met regularly during the
programming exercise in 2007 to ensure coordination during the implementation phase of the
two OPs.

The management structure was set in place in 2007 in order to prepare for the implementation
of the HRD OP, once all the necessary requirements for accreditation and conferral of
management were met.

To ensure an efficient and effective implementation of the HRD OP and fulfil the tasks, the
Ministry of Labour and Social Security has established the following five units: 1)
Programming Unit, 2) Technical Implementation Unit, 3) Monitoring and Evaluation Unit, 4)
Quality Assurance and Control Unit, 5) Information, Publicity and Technical Assistance Unit.

The functions and responsibilities of each unit have been defined and the coordinators of each
unit have been appointed. Furthermore, job descriptions for each staff were prepared.

To manage and control issues related to risks, a Risk Management Coordinator for the IPA
Management Department was assigned, and a Risk Management Committee was
established.

The first draft Implementation Manual of the HRD OP was prepared with the external
assistance of the “Support to SPO” contract.

Internal Audit Unit Department has been established in MoLSS as the HRD OS.

A training need analysis was conducted, and several internal and external trainings were
conducted in order to increase the administrative capacity of the OS.

In accordance with Article 59 of the IPA IR, a Sectoral Monitoring Committee (SMC) for the
HRD OP in Turkey was set up in 2007. The committee was composed of 44 members. 24
members were from national public institutions, 7 members from economic and social
partners, and 13 members from regional partners.

The first committee meeting for the Human Resources Development Operational Programme
was held in Ankara, on 13 December 2007, at the premises of the Secretariat General for
European Union Affairs.

Involvement of the civil society in the implementation of the OP was to be ensured by the
following activities: 1) The SMC should include members from the social partners and other
representatives of the civil society, 2) Involvement and participation of the civil society and
local administrations in the 12 NUTS II regions, including provinces and municipalities, should
be ensured. To that purpose, technical assistance may be used by the MoLSS as OS to
establish coordination mechanisms and advisory forums, 3) NGO’s and Social Partners will be
grant scheme beneficiaries on the implementation of the measures.
99
Major Development in the Implementation of the HRD-OP in 2008











Drafting of the procurement and call for proposals documents, namely terms of reference,
technical specifications for service and supply tenders, as well as guidelines for grant
applicants were prepared.
The Human Resources Development Operational Programme Implementation Manual (HRD
OP PIM) was completed.
The following institutions were designated for the implementation of the 2007 allocations of the
HRD OP: Supporting Women to Enter Labour Market (İŞKUR), Promoting Registered
Employment through Innovative Measures (SSI), Promotion of Life Long Learning (MoNE),
Promoting Social Inclusion in Turkey (SSI).
The following institutions were designated for the implementation of the 2008 allocations of the
HRD OP: Increasing Young Employment (İŞKUR), Improving the Quality of Public
Employment Services (İŞKUR), Increasing Enrolment Rates Especially for Girls in Secondary
Education (MoNE).
The preparation of the OIS for each operation was launched. Relevant organizations involved
in this process include the Ministry of National Education, the Social Security Institution, the
Turkish Employment Agency, the General Directorate on Women Status, the General
Directorate on Social Services and Child Protection, the DG on Disabilities, the State Planning
Organization, representatives of ECD and other relevant institutions, social partners and
NGOs.
The IPA Management Department and İŞKUR carried out a stakeholder analysis on
employment operations through several consultation meetings with KOSGEB, Ministry of
Science, Industry and Technology, Ministry of Agriculture and Rural Affairs, Ministry of
Education and the State Planning Organization in which what kind of activities can be
implemented in the operations were discussed. The Ministry of National Education also held a
consultation meeting with the local education directorates at provincial level. After stakeholder
analysis, for each operation, an expert and a substitute expert were appointed.
The Operation Identification Sheets prepared for 2008 allocations were updated.
Complementarity of HRD-OP components linked with other instrument.
The second SMC meeting for the HRD OP was held on 17th June 2008 in its final composition.
This composition was made permanent for the first two years (2008 – 2010). There were 24
members from the National Public Institutions and 7 members from Economic and Social
Partners, namely trade unions confederations and employer organizations. There were also
13 members from the 12 NUTSII regions and several NGOs working at the national level in
the fields related to the HRD OP. The Committee is co-chaired by the Deputy Undersecretary
of the Ministry of Labour and Social Security, as Head of the Operating Structure for the HRD
OP or Head of IPA Management Department, and a representative of the European
Commission. All committee permanent titular members and their substitutes were nominated
by their respective authorities through official letters and they were appointed by the Head of
the Operating Structure on 16 June 2008 just before the second SMC meeting.
The third SMC meeting was held on 13 November 2008. At this meeting the operations to be
funded using the 2007 and 2008 financial allocations were presented to the committee. The
minutes were finalized, taking into account comments from members, and published on the
website of the Ministry on 29 December 2008.
In terms of monitoring of grant schemes, the Central Grant Monitoring Teams (CGMT) was to
be composed of minimum 10 experts; minimum 5 experts were from the Operation Beneficiary
and when requested, the Operating Structure would provide the CGMT a maximum of 5 short
term experts assigned under the Technical Assistance Priority.
100





Regional Monitoring and Technical Assistance Teams were to be established in 12 growth
centres.
A Framework Contract under Support for European Integration (SEI) was implemented to draft
tender documents for the implementation of technical assistance priority axis.
In February 2008 two regional conferences were organized in Malatya and Gaziantep by the
SPO under the EU funded project named “Capacity Improvement in the field of Economic and
Social Cohesion” for the promotion and consultation of the SCF and OPs for all IPA
components during the programming process for the period before the start of the
implementation.
Between 11 March and 30 April 2008, the EC Delegation organised 4 regional information and
awareness raising conferences solely for the HRD OP in Trabzon, Kastamonu, Diyarbakır and
Kayseri.
A dedicated HRD OP web site was designed in Turkish and English. The domain for the
website (http://ikg.gov.tr) was received, but was not uploaded to a server by the end of the
reporting period.
Major Developments in the Implementation of the HRD-OP in 2009









The operational agreement of “Promoting Women’s Employment” was signed on the 8th
October 2009, of “Promoting Youth Employment” on the 21st December 2009, of “Promoting
Registered Employment Through Innovative Measures” on the 8th October 2009, of
“Improving the Quality of Public Employment Services” on the 21st December 2009, of
“Increasing Enrolment Rates Especially for Girls” on the 21st December 2009, of “Promoting
Lifelong Learning” on the 8th October 2009.
The implementation of the HRD OP started when the Financing Agreement was signed on
16th September 2009.
In order to ensure the complementarity between the HRD OP and other IPA Components, the
priority axes related with the ones of HRD OP were tried to be integrated to the programming
period of the Operations.
Subsequently, the fourth and fifth SMC meetings for the HRD OP were held on 29th June
2009 and 23rd November 2009. According to conclusions of the 4th SMC meeting, the
establishment and the mandate of the ad-hoc sub-committee for interim evaluation of the HRD
OP were approved.
At the fifth SMC meeting held on 23rd October 2009, the establishment of Central Grant
Monitoring Teams (CGMTs), Regional Grant Monitoring and Technical Assistance Teams
(RGMTTs), Monitoring Information System (MIS) was shared and discussed with the
committee members.
In order to monitor the projects under the grant component of the Operations (where
applicable), the process for establishing the CGMTs was initiated by the OS in 2009.
For the year 2009, overall risks for the IPA Management Department were summarized as
“follow-up the accreditation process, lack of motivation and career opportunities, work
overload for the staff, lack of administrative capacity of all relevant parties and conflict
between tasks of IPA units”.
The OS presented the Communication Action Plan to the SMC in the Fourth SMC meeting
held on 29th June 2009.
In 2009, nearly 5500 potential grant applicants were informed about the operations under the
2007 and 2008 Programming Periods. As regards the 2007 Programming Period,
approximately 3000 potential grant applicants participated in 15 Grant Information Days
organized in 15 provinces; 13 growth centers and 2 hinterlands between February-March
2009. Likewise, under the 2008 Programming Period, 13 Grant Information Days were
organized and approximately 2500 potential grant applicants were informed in July 2009.
101


RGMTTs were established with the aim of ensuring the publicity of the OP in the local level
and supporting the beneficiaries who would conduct projects in the regions. RGMTTs would
also be responsible, with respect to information and publicity, for establishing a database of
the potential applicants and organizing training programmes on the project application
process.
OS designed a dedicated HRD OP web site in Turkish and English. The domain for the
website (www.ikg.gov.tr.) was received, and uploaded to the server as of 31st December
2009.
Major Development in the Implementation of the HRD-OP in 2010









Grant contracts under Lifelong Learning Operation (LLL- I) were signed in May 2010 and
followed by the signature of the grant contacts under Promoting Registered Employment
through Innovative Measures (PRE- I), Promoting Women’s Employment (PWE- I), Promoting
Youth Employment- I (PYE- I) and Increasing Enrolment Rates Especially for Girls (IER- I).
By the end of 2010, grant projects under 5 Operations started to be implemented. Besides,
four service contracts namely Technical Assistance for Implementation of HRD OP and TA
Human Resources Development Operational Programme (TA 5.1- 28th September 2010),
Technical Assistance for Potential Operation and Grant Beneficiaries, Information& and
Publicity. (TA 5.2 & 5.3-1st November 2010), PES (22nd December 2010) and PRE (14th
September 2010) have been signed.
Within the framework of 5 grant schemes, 435 grant contracts were signed with a total budget
of 81.372.983 Euros including the co-financing contribution of the grant beneficiaries.
In 2010 two Sectoral Monitoring Committee (SMC) Meetings were held. The 6 th SMC was held
on 24 June 2010 in Van and the 7th SMC on 30 November 2010 in Hatay.
Central and regional monitoring teams (RGMTTs and CGMTs) were established under the
coordination of MoLSS.
A service contract was signed for supporting the teams. Monitoring and procurement manuals
have been prepared in compliance with the new IPA rules as well as up-to-date practical guide
and legislation.
A total of 106 experts have been assigned in the HRD OP implementation regions, especially
in the 12 provinces called growth centres. In the Centre, from the technical assistance project,
1 key grant expert and 4 short term experts have been working for supporting the monitoring
experts. For each grant scheme one CGMT was composed of at least 5 experts provided by
the Operation Beneficiary. Each CGMT is co-chaired by two representatives; one from the
Operation Beneficiary and the other from the OS. A total of 34 experts have been assigned to
central teams from the institutions such as MoNE, SSI and İŞKUR that are implementing a
grant scheme under an operation. 140 monitoring experts as state officials and 5 experts from
the technical Assistance Teams have been conducting their tasks.
Before the appointment of these monitoring experts, the CVs of all candidates were submitted
to the OS. Following the examination and scoring of the CVs the OS approved the
composition of the CGMTs and RGMTTs.
Monitoring experts assigned from public institutions were provided with a 5- day training
programme. Representatives from CFCU also gave their support within the trainings. In
addition to the trainings, a manual on the monitoring activities was given to the monitoring
experts and they have been provided with MIS accounts. A variety of works have been
conducted in order to improve the quality of monitoring activities of the monitoring experts and
increase the reliability of monitoring data. For instance, each monitoring visit report is checked
and, if needed, the monitors are asked to repeat the visit to obtain more accurate and detailed
monitoring data.
102

In 2010 two technical assistance projects have been started in order to provide support to OS
as well as the potential and actual operation and grant beneficiaries. The Technical
Assistance activities required for implementing the HRD OP are defined under three
measures: 1) Support for Programming, Management, Implementation, Monitoring, Control,
Evaluation and Dissemination Activities. 2) Support for development of absorption capacity of
final beneficiaries. 3) Information and publicity activities.
2. Major developments since 2007 at EU level
2.1 The Lisbon Strategy
Since March 2000, the EU has formulated its policies in line with the objectives of the Lisbon Strategy,
which set a framework for actions until 2010. The Lisbon Strategy aimed specifically for economic as
well as social and environmental renewal. The Strategy aimed to increase European competitiveness
by investing in a knowledge-based and highly productive society.
Noting the challenges Europe was facing from globalisation, an ageing population, and the emergence
of a worldwide information society the political leaders of EU resolved that economic and social
reforms had to take place in the context of “a positive strategy which combines competitiveness and
social cohesion”, and reaffirmed that the European social model, with its developed systems of social
protection, must underpin the strategy.
The Union “set itself a new strategic goal for the next decade: to become the most competitive and
dynamic knowledge-based economy in the world, capable of sustainable economic growth with more
and better jobs and greater social cohesion.”
This was to be achieved through a range of policies including a sound macroeconomic policy mix that
is conducive to high growth, completing the internal market, investing in people and combating social
exclusion. EU leaders pledged to aim for full employment in Europe, in a society accommodating the
personal choices of women and men.
The Lisbon Strategy intended to deal with the low productivity and stagnation of economic growth in
the EU, through the formulation of various policy initiatives to be taken by all EU member states. The
broader objectives set out by the Lisbon strategy was to be attained by 2010.
It was adopted for a ten-year period in 2000 in Lisbon, Portugal by the European Council. It broadly
aimed to "make Europe, by 2010, the most competitive and the most dynamic knowledge-based
economy in the world".
The Lisbon Strategy sets specific targets:
- An overall employment rate of 70% by 2010
- An employment rate for women of over 60%
- An employment rate of 50% among older workers
- Annual economic growth around 3%
The Council also adopted the open method of coordination between Member States, at different levels
of decision-making, as a means to achieve these ends.
103
In May 2004, as the Lisbon Strategy neared its halfway point, the European Commission (acting on
the conclusions of the March 2004 European Council) set up a High-Level Group of Independent
Experts chaired by former Netherlands Prime Minister Wim Kok, to make an independent assessment
of progress. The 13 experts represented the different stakeholders in the strategy. The group met six
times and submitted its report, “Facing the Challenge”, to the Commission and Council on November 4
2004. Its aim was to prepare for the European Council’s mid-term review in March 2005.
It concluded that the EU was very unlikely to meet its 2010 goals, chiefly due to a lack of determined
political action. It highlighted an overloaded agenda, poor coordination, and conflicting priorities. It also
pointed out that structural reform has become a codeword for deregulation and weakening workers’
rights, and noted that policies should, instead, help workers to address structural change (investing in
skills and productivity instead of deregulating labour markets). It also underlined the vital importance of
aggregate demand management to exploit fully Europe’s growth potential, and argued that the
Stability Pact was an obstacle to this.
Furthermore, the Kok Report stressed the importance of sustaining the European social model, and
advised against copying the US system of minimal social welfare. The High Level Group’s proposals
focused on the need to communicate better with EU citizens, and obtain their support for the reform
process - something the ETUC has repeatedly emphasised. Importantly, the report stressed the role
that social dialogue and the social partners, both at national and European level, can play in delivering
the Lisbon objectives of high non- inflationary growth, more and better jobs and strong social
cohesion.
In November 2009 the Deutsche Bank wrote:
“The Lisbon Agenda of the past decade disappointed expectations that it would drive reform,
partly because some of its targets were inconsistent but mainly due to the member states’ lack
of reform commitment. Under the Open Method of Coordination the only sanctions available to
the EU were “soft” options such as peer pressure. Whilst this did strengthen the hand of
governments willing to reform, it was often an inadequate spur to those who had tired of the
process.”
[Deutsche Bank Research, EU Monitor 70, November 2009, p. 1]
In June 2010 “The Europe 2020 Strategy” plan for economic renewal was launched, replacing the
Lisbon strategy (2000-10). By 2020, the new strategy aims to guide Europe's economy out of the
economic recession (which began in 2008) and to enable a high quality of life whilst preserving
Europe's Social Model, raising employment, productivity and social cohesion.
2.2 The Europe 2020 Strategy
The Europe 2020 Strategy has established the following global targets:
1. Employment
 75% of the 20-64 year-olds to be employed
2. R&D / innovation
 3% of the EU's GDP (public and private combined) to be invested in R&D/innovation
3. Climate change / energy
 greenhouse gas emissions 20% (or even 30%, if the conditions are right) lower than
1990
 20% of energy from renewables
 20% increase in energy efficiency
104
4. Education
 Reducing school drop-out rates below 10%
 at least 40% of 30-34–year-olds completing third level education
5. Poverty / social exclusion
 at least 20 million fewer people in or at risk of poverty and social exclusion
The Commission has further defined seven flagship initiatives to catalyse progress under each priority
theme:
1. "Innovation Union" to improve framework conditions and access to finance for research and
innovation so as to ensure that innovative ideas can be turned into products and services that
create growth and jobs;
2. "Youth on the move" to enhance the performance of education systems and to facilitate the
entry of young people to the labour market;
3. "A digital agenda for Europe" to speed up the roll-out of high-speed internet and reap the
benefits of a digital single market for households and firms.
4. "Resource efficient Europe" to help decouple economic growth from the use of resources,
support the shift towards a low carbon economy, increase the use of renewable energy
sources, modernise our transport sector and promote energy efficiency.
5. "An industrial policy for the globalisation era" to improve the business environment, notably for
SMEs, and to support the development of a strong and sustainable industrial base able to
compete globally.
6. "An agenda for new skills and jobs" to modernise labour markets and empower people by
developing their of skills throughout the lifecycle with a view to increase labour participation
and better match labour supply and demand, including through labour mobility.
7. "European platform against poverty" to ensure social and territorial cohesion such that the
benefits of growth and jobs are widely shared and people experiencing poverty and social
exclusion are enabled to live in dignity and take an active part in society.
Similar to the Lisbon Strategy the Europe 2020 Strategy relies on two pillars: the thematic approach
outlined above, combining priorities and headline targets across EU combined with country reporting
in which the Member States develop their strategies to return to sustainable growth and public
finances all principles given by so-called Integrated guidelines to cover the scope of EU priorities and
targets.
The two-level system appears important. The Supra-national vision, supported by guidelines as well
as a unified reporting system, facilitates and drives Country-specific recommendations within the
Member States. In addition “policy warnings” could be issued in case of inadequate response. We
shall return to the system below in this report.
2.3 The EU Employment Guidelines and the Broad Economic Policy Guidelines
The employment guidelines proposed by the Commission and approved by the Council, present
common priorities and targets for the national employment policies. They have been in an integrated
package with the Broad Economic Policy Guidelines since 2005.
The Broad Economic Policy Guidelines underneath the Europe 2020 Strategy describe an overall
economic policy guidance to take the form of guidelines addressed to Member States. These
guidelines enable the economic policies of the Member States to be coordinated in order to achieve
joint objectives.
105
The Broad Economic Policy Guidelines are described in Council Recommendation 2010/410/EU of 13
July 2010 on broad guidelines for the economic policies of the Member States and of the Union
[Official Journal L 191 of 23.7.2010]. The Guidelines should be taken into account by Member States
in the implementation of their economic policies and the development of national reform programmes.
The Broad Economic Policy Guidelines comes under four main pillars:
1. Sustainability of public finances and macroeconomic stability
2. Smart growth
3. Sustainable growth
4. Inclusive growth
While the Broad Economic Policy Guidelines set the scope for recommendation to national policies in
broad sense, the Employment guidelines of the Europe 2020 Strategy “should enable the EU to
achieve smart, sustainable and inclusive growth in the next ten years. In this Recommendation, the
Council also notes the importance of the sustainability of Member States’ public finances and their
macroeconomic stability.” [see: The EU Commission Home Pages]
Since October 2010 the current EU Employment guidelines are as follows:
 Increasing labour market participation of women and men, reducing structural unemployment
and promoting job quality
 Developing a skilled workforce responding to labour market needs and promoting lifelong
learning
 Improving the quality and performance of education and training systems at all levels and
increasing participation in tertiary or equivalent education
 Promoting social inclusion and combating poverty.
3 Coherence in objectives
3.1 The Strategic Coherence Framework
In order to provide a frame of reference for RC OP and HRD OP, the Strategic Coherence Framework
(SCF) has been prepared under the coordination of the Strategic Coordinator, SPO, [now Ministry of
Development] by including the contributions of relevant authorities, priorities defined in main strategy
documents and contributions of the Commission.
As a major strategic document, the SCF takes into account the priorities of the Republic of Turkey and
the European Union as stated in major policy documents, especially in the Multi-annual Indicative
Planning Document (MIPD). It aims to provide coherence and consistency between the two
operational programmes for the 2007 – 2013 period. HRD OP has been prepared in line with the SCF.
In order to provide coherence with the SCF, several consultation meetings have taken place between
the Operating Structure (OS) and the Strategic Coordinator.
The key challenges defined in SCF are:
 Low labour participation rate for women
 High unemployment especially for young
 Problems stemming from decrease of agriculture sector
 Unregistered employment
 Low enrolment rates of girls
 Late enrolments, dropouts and problems of girls’ access to education
 Low quality of education
106



Mismatches of the labour market and education system
Inefficient social assistances and social services
Problems of access to employment, education and social services for disadvantaged persons
Accordingly, the SCF sets out four main priority axes that also constitute the basis for HRD OP. These
axes are,
 Attract and retain more people in employment, particularly by increasing labour force
participation of women, and decrease unemployment rates, especially for young people.
 Enhance investment in human capital by increasing the quality of education improving the
linkage between education and labour market and raising and enrolment rates at all levels of
education especially for girls.
 Increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting
lifelong learning and encouraging investment in human resources by enterprises and workers.
 Promote an inclusive labour market with opportunities for disadvantaged persons in terms of
their sustainable integration into the labour force and combat all forms of discrimination in the
labour market.
The main priority axes established by the SCF have been incorporated in and adapted by the HRD
OP, which again has establish specific objectives, measures, indicators and targets in alignment
thereof. It appears accordingly fruitless to make further comments to the coherence between the SCF
and the HRD OP, since the HRD OP is based upon the SCF platform.
Throughout the Interim Evaluation the Interim Evaluation TAT has been assured about the
communication and control systems between the stakeholders involved in the two central documents,
which communication lines appear both solid and comprehensive. The TAT has no further comments
to add to the correspondence between the central planning documents and the HRD OP. For further
reference on links between the two documents we kindly refer to the ex-ante evaluation.
3.2 Coherence between HRD OP and the main EU strategy
Since the Lisbon Strategy has been replaced by the Europe 2020 Strategy it appears less fruitful to
analyse the HRD OP’s coherence with the Lisbon Strategy. Accordingly, let us turn to the Europe 2020
Strategy.
In this context of evaluating the coherence between the Turkish HRD OP and the Europe 2020
Strategy described above, the strategic framework of the Europe 2020 Strategy in which only global
targets have been set allowing the Member States to define the specific national objectives, measures,
indicators, and targets, the matter of coherence becomes slightly blurred; It would be far more
consistent to compare the HRD OP objectives and targets with the national agendas of the Member
States. Also since the Europe 2020 Strategy does not exclusively address Human Resource issues.
In return to the issue of coherence between the Turkish HRD OP and the Europe 2020 Strategy the
very global and cross-sectoral issues established as targets which not necessarily focus exclusively on
Human Resource Development the issue of coherence becomes vague. However, and by noting that
all target areas necessarily must include elements which relates to Human Resources, though only
indirectly, the Europe 2020 Strategy targets mentioned above at least provides some targets which
relates directly to the Human Resources Development; that is:
107
1. Employment
a. 75% of the 20-64 year-olds to be employed
2.
Education
b. Reducing school drop-out rates below 10%
c. at least 40% of 30-34–year-olds completing third level education
3.
Poverty / social exclusion
d. at least 20 million fewer people in or at risk of poverty and social exclusion
And similar the Europe 2020 Strategy sets flagships which more or less directly relate to the Human
Resources Development; that is:
1. "Innovation Union" to improve framework conditions and access to finance for research and
innovation so as to ensure that innovative ideas can be turned into products and services that
create growth and jobs;
2. "Youth on the move" to enhance the performance of education systems and to facilitate the
entry of young people to the labour market;
3. "An agenda for new skills and jobs" to modernise labour markets and empower people by
developing their of skills throughout the lifecycle with a view to increase labour participation
and better match labour supply and demand, including through labour mobility.
4. "European platform against poverty" to ensure social and territorial cohesion such that the
benefits of growth and jobs are widely shared and people experiencing poverty and social
exclusion are enabled to live in dignity and take an active part in society.
Table 3 summarizes the main focus areas of the two. It is worth to note that the Europe 2020 Strategy
is highly generic leaving the measures and the national targets to each single Member States; it is
accordingly up to the single Member States to comply with the EU targets. In some (if not most) EU
Member States the targets is further transposed to regional targets and actions where the national or
central level is left only to assure a generic national target. In other words, it tend to be quite
impossible to describe the often various different ways the Member States have chosen to comply with
the supra-national guidelines and targets.
However, even that feature could be seen as a doable future way for Turkey as well; after all the
situation in the various regions of Turkey differ to some extent, which could argue for more regional
flexibility and a stronger regional, perhaps even local, involvement in target setting and monitoring
thereof in compliance with regional objectives. The central level should then just assure that the
overall national targets are met, subject to the situation and the incentives taken in NUTS 2 regions
not participating in the HRD OP.
The last comment on changing the overall structure of the HRD OP, including its management,
however, appear to be slightly out of scope for the Interim Evaluation of the HRD OP 2007-2009
programme, mainly since the Interim Evaluation has found weaknesses on regional level (see
Thematic Report II).
Table 3. Coherence between the EU Framework and HRD OP
EU Framework
Employment target from Europe 2020 Strategy:

75% of the 20-64 year-olds to be employed
Employment Guidelines:
HRD OP
Employment:
PRIORITY AXIS -1: To attract and retain more people
in employment, particularly by increasing labour force
participation of women, and decrease unemployment
rates, especially for young people
108


EU Framework
Increasing labour market participation of women
and men, reducing structural unemployment and
promoting job quality
Developing a skilled workforce responding to
labour market needs and promoting lifelong
learning
HRD OP
Specific Objectives:

Promote labour force participation and
employment of women, including those formerly
employed in agriculture.

Increase youth employment.

Promote registered employment.

Improve public employment services
PRIORITY AXIS – 3: To increase adaptability of
workers, enterprises and entrepreneurs, in particular by
promoting lifelong learning and encouraging investment
in human resources by enterprises and workers
Specific Objectives:

To promote Life Long Learning (LLL) Opportunities
under a LLL Strategy

To improve quality of non-formal trainings

To increase adaptability of employees

To increase adaptability of employers in SMEs

To promote well-functioning of the National
Qualifications System
Education targets from Europe 2020 Strategy:

Reducing school drop-out rates below 10%

at least 40% of 30-34–year-olds completing third
level education
Employment Guidelines:

Improving the quality and performance of
education and training systems at all levels and
increasing participation in tertiary or equivalent
education
Poverty reduction / social exclusion targets from
Europe 2020 Strategy:

at least 20 million fewer people in or at risk of
poverty and social exclusion
Employment Guidelines:

Promoting social inclusion and combating poverty
Education:
PRIORITY AXIS-2: To enhance investment in human
capital by increasing the quality of education, improving
the linkage between education and the labour market,
and raising enrolment rates at all levels of education,
especially for girls
Specific Objectives:

To increase enrolment rates particularly for girls at
all levels of education and vocational training

To increase the quality of education especially in
vocational education and training

To develop innovative approaches to improve
linkage between education and labour market

To promote a “partnership approach” to modernize
VET system
Poverty reduction and Social Inclusion:
PRIORITY AXIS – 4: To promote an inclusive labour
market with opportunities for disadvantaged people,
with a view to their sustainable integration into the
labour force and combat all forms of discrimination in
the labour market
Specific Objectives:

To facilitate sustainable integration of the
disadvantaged into the labour market

To improve the functioning and coordination of
institutions and mechanisms in the field of labour
market and social protection, particularly in order
to facilitate the integration of disadvantaged
persons into the labour market.
109
It appears obvious that the HRD OP which covers objectives within the Human Resource areas, which
includes Employment, Education and Lifelong Learning, as well as Social Inclusion tangents the
objectives of the EU Framework for a European Strategy as laid down in the Europe 2020 Strategy
and further accentuated towards the Human Resource issues in the Employment Guidelines. After all,
the subject is the same and to some extent the challenges are identical, since EU faces major
challenges partly from the economic turmoil of the global economic crisis and partly faces increasing
challenge from other more rapidly expanded countries mainly in Asia and notably China and India.
Similarly it appears obvious that the target frames for the two frameworks might differ just as the intraEU targets and objectives differ to a certain degree; after all the specific ways to combat the
challenges facing the individual Member States as well as Turkey cannot be identical as the starting
points differs significantly.
Accordingly, when it comes to evaluation of the coherence between the Europe 2020 Strategy, which
by its very nature is a global EU strategy where it is up to the individual Member States to fill the
strategy out with specific objectives and targets, and the Turkish HRD OP the question appear more to
be: Why be concerned about coherence if the broad perspectives are the same? However, that
question is not part of the Report’s target.
The question of coherence between the EU Framework for HRD and the Turkish HRD OP leaves only
one answer: Yes, there is a large extent of coherence, since the objectives of EU and Turkey are
highly similar and so are the target areas to meet the challenges, and since both EU and Turkey has
understood the agenda and taking it into account.
We could perhaps as a final word note that the HRD OP does not specifically mention Poverty
Reduction as a main focus area contrary to the Europe 2020 Strategy, which with reference to e.g. the
Socio-Economic Analysis in a Chapter above may appear quite strange; after all Turkey faces far
more basic and important problems than EU and has not the same means to pursue more income
equality between regions.
However, the lack of focus on the topic of Poverty Reduction in the HRD OP is a matter of the degree
explicitly formulating an objective which basically works as the fundamental principle of the complete
Operational Programme; the issue of Poverty Reduction is definitely included both in the selection of
Regions to take part in the HRD OP as well as deeply described more or less explicitly in the rationale
of each of the priority axes. And to some extent we have to give credit to the Turkish: It appears trivial
to state the obvious; rather focus on the main source for change.
The overall conclusion is accordingly that there appear to be a fairly obvious coherence between the
EU Framework and the HRD OP.
110
Annex 3. Review of the results of ex-ante evaluation
The findings of the ex-ante evaluation are summarized in the updated second version of the HRD
Operational Programme under a set of headlines. The Interim Evaluation has followed up on the main
findings and created some single comments. Of course, time has elapsed and not all of the
recommendations given by the ex-ante evaluation are equally relevant anymore.
1. Partnership Approach
See Thematic Report on Horizontal Issues.
2. The study of the Current Situation
The ex-ante evaluators note a valid point of providing reasons to undertake solid measures in Human
Resource Development within the less developed regions, not least with respect to Lifelong Learning,
which provision should be given by analyses of the current situation. However, some very important
arguments appear to be missing in the ex-ante evaluation, and the same goes for theoretical points of
methodology.
The point, as the Interim Evaluation understands the ex-ante evaluation, is to provide reasons to
undertake solid measures in Human Resource Development within the less developed regions of
Turkey. In more plain words the question is: “Why do we do all that effort to support the Human
Resource Development in the regions in question?” Hence the background study, which not
necessarily is the same as indicators of the performance of the programme (though often tightly
related).
And in that respect we often runs into methodological challenges: How to prove that we need to invest
in Human Resources? From a scientific perspective the answer to that question often blows in the
wind, though we are very clear on the impact of our effort.
The challenge in understanding why becomes even more severe and visible when it comes to Lifelong
Learning; how to provide reasons for continuously upgrading the existing labour force? Exactly where
can we see that the current situation is inefficient, perhaps inappropriate, and needs to be upgrade or
perhaps even call for complete reforms of the educational system?
It ought to be obvious that these questions can be difficult to provide solid answers to not least from a
scientific perspective. And often, notably in advanced mature EU Member States we hardly ever
address the questions in the basic reports, simply because our experience knows actions and
measures are needed; it has become a common understanding, a common knowledge 27. Most
countries embark on systems based upon Barometer Surveys of labour shortages combined with
follow-up measures such as Training Needs Analysis targeted to specific sectors and specific
occupation within specific locations, but in general these tools hardly ever provide the fundamental
reason of “why?”28.
27
This is basically not correct; the point is merely that it takes some solid scientific studies to prove the solid
reason for the investment in Lifelong Learning, due to the immense number of other factors which are involved in
the basic questions raised here.
28 Agreed, the discussion tends to be slightly theoretical and advanced here. Training Needs Analysis, TNA, has
some very good features and serves a purpose if done in accordance to the theoretical limits within the method.
However, and as recognized by all major scientist within this area, employers do not necessarily respond in
accordance to the findings of the TNA; it tends to be obvious at least to understand that some employers rather
prefer to hire new, qualified labour than invest in training of the existing employees.
111
Accordingly, the background studies of the current situation should be able to make sure that all
readers of the reports understand why we invest so much effort (and means) in continuously
developing human capital, which otherwise, in case the free market forces worked perfectly, should be
solved by default. We need reasons, that is, and hence the issue raised by the ex-ante evaluators on
at least findings some sort of proof for the need for solid measures.
But from here some link appears to be broken from a theoretical perspective. True, Priority Axis 3 and
its focus on Lifelong Learning could need some arguments, but basically it appears far too limiting just
to “add data regarding the productivity of the Turkish economy, notably economic performance data
for Turkish SMEs” as noted in the recommendation.
The reason for the broken link in terms of adding data on SME should be very obvious; though
presumably constituting somewhere between 85 and 95 of total enterprises (as in almost all other
countries) and presumably a significant fraction of total employment, SMEs are by no means the total
labour market. In many cases we could easily argue that establishing in Human Resource
Development plans is far more crucial in larger enterprises for various highly justified reasons 29.
True, SMEs may not have the same capacity to invest in Lifelong Learning as large enterprises, and
similarly it appears obvious that today’s SMEs may rightfully become tomorrow’s LE (large and main
enterprise). But still the labour market is (hopefully) the complete labour market which accrues to the
question of Lifelong Learning as well.
In other words, it appears less reasonable to incorporate studies of performance among SMEs alone
in the background studies to support public investments in Lifelong Learning; we need to do studies of
the complete labour market for obvious reasons. And that goes with respect to Priority Axis 3 as well.
However, and this is where the real clarification enters, Priority Axis 3 has a Specific Objective 4 “To
increase adaptability of employers in SMEs” as well as a defined target group of “Employees in SMEs
in specified sectors in RC-OP” under Specific Objective 3 “To increase adaptability of employees”
which obviously necessitate arguments, and hence the recommendation made by the ex-ante
evaluators.
It appears from the above that the designers of the HRD OP did not understand the basic rationale; of
course, the background studies of the HRD OP should provide solid arguments for the targets in
question, which arguments are not done by referring to another OP. In particular, and to refer to the
rationale provided in this context, the stakeholders need to understand why the exclusive focus on the
SMEs, which do not appear evident from the HRD OP.
Accordingly, the recommendation remains in this very Interim Evaluation Report, and due to a need to
allow the stakeholders to understand the rationale behind the exclusive focus on SMEs added the
note that:
The background study of the current situation should contain studies of production
output on economic sectors of both SMEs and large enterprises
However, the phrase does not provide the rationale for investment in Lifelong Learning. In that respect
we need to use measures of productivity.
29
In many countries large enterprises still constitute the backbone if not the complete bearing platform of the
economy in question which makes it obvious to ensure that these enterprises understand and adapt to global
trends and technological changes in time. Furthermore, it appears similarly obvious that the core well-established
enterprises, notably in less-developed, perhaps even poor, regions, should act as driving motors / role models for
investments in business plans and accordingly HRD on micro level.
112
Productivity is not production but production (output or value added, rather) per employee. The idea of
the measure, which agreed is complex, is that it contains vital elements related to issues of
technological input (both physical and human technology (education, that is)) and thus the level of
advance. Turning the measure into growth rates is measures further progress and accordingly
development towards a more advanced economy. The presentation of the measure will further be
described in the evaluation of the indicators below.
In the Socio-Economic Study above the measure is used on a general scale for all types of
enterprises. The reason is that it is not possible to obtain the raw data needed for the measure on
sizes of enterprises.
Accordingly, the idea of retaining the recommendation from the ex-ante evaluation appears to be weak
from a first view: Production provides no rationale for investment in Human Resource and productivity
cannot be measured on size of enterprise. However, the logic comes of course in two steps: First, we
have to understand the size (and growth) of the output from SMEs relative to total enterprises, and
then subsequently we have to understand the growth in productivity. The two should obviously be
connected in the background study.
We shall, as mentioned, return to these points in the study of current indicators below.
With respect to the inclusion of target groups for social inclusion (recommendation 2 above) the
adjustment of the HRD OP appears appropriate and the TAT found no need to add further comments.
3. The Effectiveness of the planned strategy
The Interim Evaluation has no comments to the steps taken in the revised HRD OP except for the
evaluation of the indicators and their relevance in Chapter 4 and Annex A4.
4. Indicators
The ex-ante evaluation has a long range of recommendations to the selected indicators. The Interim
Evaluation have made an extensive study on the indicators as well to which we refer. With respect to
the findings of the ex-ante evaluation as well as the steps taken to incorporate the recommendations,
the interim evaluation has found no reason to make additional comments to the general study of
Indicators.
5. Sustainability of Results
The Interim Evaluation has investigated the function of the project promoters and found no reason to
make further comments. With respect to Mainstreaming of activities and the function of the
Programme Management Unit of the Operating Structure it is the general view of the Interim
Evaluation that the process has to be slightly more mature in order to assess the effect30.
6. Absorption Capacities
Absorption capacities are explicitly dealt with in Chapter 4.
30
Most of the project was just started at the cut-off date of April 30. In addition the monitoring of the progress of
the projects is subject to time-lags in reporting making is almost impossible partly to evaluate the mainstreaming
activities and their effect and partly to obtain a solid platform for documentation.
113
7. The ex-ante evaluation’s conclusions and recommendations regarding HRD
OP implementation
In the ex-ante evaluation report the overall recommendations are summarized in the following table
6.1.7. The table further describes the response in the revised HRD OP.
Table 6.1.7 Summarizing table of the overall recommendation of the ex-ante evaluation and the
subsequent response
Ex Ante Evaluation Report
HRD OP revision and response

The foreseen division of units within the
HRDOP OS should be revised and
streamlined
in
the
view
of
the
recommendations on IPA structures
formulated by the ESC project. Thus,
instead of five units there should only be
three
(Programming,
Monitoring
&
Evaluation, Technical Assistance &
Implementation, and Quality Assurance &
Control).

The foreseen structure has been developed in line with
the consultation process with the NAO which will be
responsible for the national accreditation. Regarding the
functions and responsibilities of each unit the OS does
not agree that Programming, Monitoring and Evaluating
should gathered under one unit as well as technical
assistance and Implementation Units.

Functions and staffing of the units’
employees have to be described in detail
at least in the Implementation Manuals

In line with the ongoing training and work load analysis
such information will be integrated on the manual and
OP.

The institutions invited for membership to
the Sectoral Monitoring Committee should
be listed in detail.

The institutions invited for membership are already
mentioned under the partnership section. In order to
avoid duplication, in the implementation section a
different approach has been followed and they have
been stated as groups.

The setup of the Monitoring Committee
should ensure the representation of
regional and local actors from the
targeted 12 NUTS II regions and 15 growth
centres. This does not mean that every
target area should be represented, but the
programme should avoid focussing only
on central government and Ankara-based
stakeholders.

The regional and local actors will be able to take part in
the Monitoring Committee meetings with the new set up

The gender balance within the Monitoring
Committee should be ensured by the OS
by setting up clear regulations. Rules of
procedure have to be set up as soon as
possible.

Gender balance issues
Implementation manuals.

The steering functions of the MIS for the
achievement of programme objectives
should be reflected within the OP. It is
also needed to describe the cooperation
with the other OSs in general and SPO in
particular.

Such details will be dealt after their finalisation in the
Implementation manuals.
will
be
dealt
in
the
114
Ex Ante Evaluation Report
HRD OP revision and response

The establishment of a functional
independent evaluation team should be
avoided in order not to produce
overlapping tasks with the Evaluation
(sub-) Committee.

The establishment of a functional independent evaluation
team was a recommendation of the Commission. In
order to

The roles and functions of institutions
supporting the OS in the implementation
of HRDOP should be formulated in detail
with a view to the transfer to accredited
Implementing Bodies.

Such details will be dealt after their finalisation in the
Implementation manuals.

The Selection Committee should be
organised as a sub-committee of the
Sectoral Monitoring Committee.

Such details will be dealt after their finalisation in the
Implementation manuals.

SPO should urge MoLSS to develop a
transparent delivery mechanism and
describe it in detail in chapter 5 of
HRDOP.

Such details will be dealt after their finalisation in the
Implementation manuals.

MoLSS should enhance the
communication strategy with elements
directed at the final beneficiaries’ active
participation and develop a
mainstreaming strategy in order to ensure
the impact of HRDOP.

Such details will be dealt after their finalisation in the
Implementation manuals.
The Interim Evaluation has noted that the Operating Structure has formed seven (7) units (see points
1 and 2). The observations and assessment of the Operating Structure follows from our study of the
management structure elsewhere in the Interim Evaluation Report. With respect to the listing of SMC
Members the TAT has no comments.
With respect to the recommendation from the ex-ante evaluation on the set-up of the SMC, it appears
less obvious to the Interim Evaluation that the “Monitoring Committee should ensure the
representation of regional and local actors from the targeted 12 NUTS II regions and 15 growth
centres”. In general, we should expect a decent regional distribution of members and subsequently
involvement and ownership, but it is a general illusion that representation alone (e.g. quotas-like
solutions) generate any additional value to the quality of the committee.
Running the risk of sending a political incorrect message, the Interim Evaluation would rather take a
far more pragmatic view on the question of representation; basically, and from a scientific or statistical
viewpoint, the main factor in the HRD OP is not the regions in question, but the target groups within
the regions. The pragmatic view would therefore rather favour the following priorities in the constitution
of the SMC:
1) A solid Partnership Approach based upon a tripartite representation 31
2) Some solid representation from regional level based upon objective grounds of quality,
knowledge and perhaps even objectivity (rather than subjectivity)
3) A high degree of representation from both genders
I.e. one-third from representing Employer’s, one-third representing employees (Trade Unions), and one-third
representing public administration and governmental / public interests. The last would include educational
institutions as well.
31
115
The illusion is that representation alone is able to outweigh competence, experience and will-power in
terms of quality of the work being done. And that illusion is repeated in point 5 on gender balance.
There is simply no scientific proof to support that equal representation of both gender works better
than a representation of the most competent person irrespectively the gender in decision-making
bodies. We might form idealistic hypothetical illusions on ethical reasons based upon arguments of
humanity, fairness and equality, but that does not necessarily indicate that things work towards a more
equal, and a more democratic, fair world; if those taking part in the process have limited knowledge
and interest in the topics discussed the process might well have no impact.
Human Resource Development is the core interest of the real actors of the labour market, including
not least those representatives who can establish changes and ensure solid reforms. Especially when
it comes to representatives of employees it appears slightly obvious to the interim evaluators that the
representatives of the employees, the Trade Unions of Turkey, have their strongholds in Ankara and
Istanbul. In that case it tend to be an illusion, a token of a non-realistic dream world, to recommend
actors from regional level to be selected prior to the real actors; that would only decrease the
functionality of the SMC, though it might look better on paper for ethical reasons.
It goes without saying that participation of regional and perhaps even local stakeholders will be
essential for obtaining viewpoints of the regional and local agendas (which soon might differ from the
national agendas in Ankara), but in that respect the focus should be on engaging representatives
which have experience with working with HRD issues from local/regional level. In other words, the
focus should be on quality more than representativeness; the recommendation should rather be in the
direction of ensuring knowledge and know-how from a practical implementation point of view.
The political incorrect message should, however, not be exaggerated and dramatized; of course the
stakeholders should ensure solid representation from regional and local level and of course that
should be incorporate as a working principle within the constitution of the SMC. And similarly with the
issue of the gender balance. The main point raised by the Interim Evaluation is rather to embark on
objectives towards obtaining the goal of representation, rather than force it through without solidity.
Of course women should have their seat in the SMC since the gender issue is a real topic in HRD.
And of course representatives from regional level should be ensured to provide knowledge of regional
and local agendas and provide ideas to what works in terms of implementation and what would never
work.
In that respect, the interim evaluation would prefer not to set rigid quotas which would distort and
disfavour the work of the SMC but rather pursue something far more pragmatic in its
recommendations:
1) Ensure a pragmatic and doable approach to the issue of representation which has its outset in
a realistic and objective background based upon knowledge of the Turkish environment
2) Ensure objectives and perhaps even targets in representation. That could be:
a. e.g. a target of 50/50 representation between regional and central level in x years; and
b. e.g. a target of xx/xx representation between the two gender in x years
The Interim Evaluation has found no reason to comment on the remaining recommendations noted by
the ex-ante evaluation.
116
Annex 4. Study of the HRD OPs Effectiveness and Efficiency
1 The use and financial allocation of financial assistance and the financial
management of the budget
Background information
Financial Allocation for HRD OP from EU
The Multi Annual Financial Framework (MIFF) provides 158.7 million Euros to the HRD OP for the
period 2007-2009 as follows:
Year
Budget (MEUR)
2007
50.2
2008
52.9
2009
55.6
Total
158.7
National Co-financing for HRD OP
The Community contribution may not exceed the ceiling of 85 % of the eligible expenditure at the level
of the priority axis. No operation shall benefit from a higher co-financing rate than the one relating to
the priority axis concerned. A national co-financing rate of 15 % and a Community co-financing rate of
85 % will be applied for all priority axes of the HRD OP.
The HRD OP budget with national co-financing is as follows:
Year
Budget (EUR)
2007
59 058 824
2008
62 235 295
2009
65 411 766
Total
186 705 885
Commitment and Disbursement Data
The commitment and disbursement date of the HRD OP on July 31, 2011 is presented below:
Commitment data in EUR
Commitment of EU contribution
91.824.245
Commitment of national co-financing
16.204.278
Total commitment
108.028.523
Commitment rates
Commitment rate of EU contribution
66.84%
Commitment rate of national co-financing
66.84%
Total commitment rate
66.84%
117
Disbursement data in EUR
Disbursement of EU contribution
55.394.668
Disbursement of national co-financing
9.775.529
Total disbursement
65.170.197
Disbursement rates as percentage of commitments
Disbursement rate of EU contribution
60.33%
Disbursement rate of national co-financing
60.33%
Total disbursement rate
60.33%
Disbursement rates as percentage of total budget
Disbursement rate of EU contribution
40.32%
Disbursement rate of national co-financing
40.32%
Total disbursement rate
40.32%
118
2. Review programme outputs and results in relation with the expenditures
incurred at priority and measure levels; in the light of financial and physical
progress
Statistical data32
Available statistical data for the grant scheme Promotion of Life Long Learning:
General data
Total budget in EUR
5.620.504
Total commitment of EU contribution and
national co-financing in EUR
4.803.264
Total number of grants
45
Distribution of trainees
Situation
1.
2.
3.
4.
5.
6.
No of Trainees
%
In progress
Completed
Failed by absenteeism
Left by his/her own will
Failed / unsuccessful
Forced to leave by administration staff
2.898
1194
54
52
51
1
68.19
28.09
1.27
1.22
1.20
0.02
TOTAL
4.250
99.99
Distribution of trainees by duration of training
Duration of Training
1.
2.
3.
4.
5.
N
%
0 - 25 days
25 - 50 days
50 – 75 days
75 – 100 days
More than 100 days
2.601
593
443
63
550
61.20
13.95
10.42
1.48
12.94
TOTAL
4.250
99.99
Distribution of trainees by work experience before training
Worked Period
1.
2.
3.
4.
5.
N
%
Never worked
Worked 0 – 90 days (SSI)
Worked 90 – 180 days (SSI)
Worked 180 – 360 days (SSI)
Worked more than 360 days
2.017
825
230
383
795
47.46
19.41
5.41
9.01
18.71
TOTAL
4.250
100.00
Distribution of trainees by employment situation
Employment Situation
1.
2.
N
%
Unemployed
Working
2.359
995
70.33
29.67
TOTAL
3.354
100.00
32
Please note that the following should not be considered as a final analysis, but just as a sample. While LLL is
the first grant scheme that has been completed under the HRD OP, the beneficiaries were in the 3 months phase
of preparing the final report and finalising administrative side of the projects (including MIS updates) at the time of
drafting the report.
119
Comments to the statistical data:
- The total budget is based on the data from the MIS
- The total commitment of EU contribution and national co-financing follows the 85/15 ratio and
the data is obtained from the financial reporting system for IPA. The final disbursement data is
not yet available, but it may not exceed the contracting amount. All calculations should be
based on the disbursement data, but for provisional and therefore inaccurate results, the
commitment data is used.
- The number of grants is based on the data from Europeaid website and from MIS
- The data on the number of trainees is obtained from MIS. However, the data has not been
updated yet as the majority of trainees are still identified as “in progress”. The assumption in
the following calculations is that the success rate will remain identical with rest of the trainees.
- The rest of the data is obtained from MIS
Based on the information above, it is still premature to perform an assessment programme outputs
and results in relation with the expenditures incurred at priority and measure levels; in the light of
financial and physical progress even in case of the grant scheme that has been completed first.
However, some conclusions based on provisional incomplete data is given below.
Analysis
One of the possibilities to measure outputs and results in relation with the expenditures incurred at
priority and measure levels; in the light of financial and physical progress is to analyse the total budget
spent on a trainee. In case of the Life Long Learning project the data is the following:
Provisional total budget per trainee in EUR
1.322
Provisional total EU contribution and national
co-financing per trainee in EUR
1.130
Estimated provisional total
successful trainee in EUR
per
1.497
Estimated provisional total EU contribution and
national co-financing per trainee in EUR
1.280
budget
The calculations above are based on provisional statistical data and are not thus accurate. It was
assumed that the success rate of the trainees that have not completed the studies remains the same
as the success rate of the trainees that have finished the studies. In reality, the success rate may be
lower. Also, in budget calculations the use of the contracted amount means that the final funds spent
will not be higher, but most probably will be 5-20% lower. The duration of the courses varies, but the
majority of the courses have a duration of less than 25 days. At this point of time, there can not be any
data available to which extent the trainings contributed to the employability of the trainees, i.e, how
many of the trainees will be employed and as a result of the knowledge and skills obtained in the
trainings under this grant scheme.
The amount of provisional data and variables makes it impossible to make a conclusive assessment at
this point of time. However, considering that the grant projects have several secondary objectives, the
average amount spent on trainees on average is reasonable, considering that almost 40% of the
trainings have duration of more than 25 days.
Since it is difficult to assess even a measure that involves a grant scheme that has been completed
first under the HRD OP, assessment of other measures would not be feasible at this point of time.
120
3. Examine the efficiency of Operation Selection Criteria
Background information.
Selection criteria for Measure 1.1 are listed below.
The HRD OP includes the following selection criteria and eligible actions:
According to the HRD OP, actions which may be funded are:
•
Labour market analyses in required regions and/or sectors with a view to guide the operations
to be implemented under this measure.
•
Activities to diminish obstacles hindering target group of women’s participation in the labour
market.
•
Active labour market measures for target group of women out of labour force, as well as those
formerly employed in agriculture.
•
Support to childcare and elderly care facilities, particularly for the target group of women
taking part in above mentioned actions.
•
Actions to provide child and elderly care facilities by training target group of women on such
care services
•
Actions to provide carrier guidance service to target group of women
•
Supporting target group of women entrepreneurship through consultancy services and training
The HRD OP refers to the following selection criteria:
•
Demonstration of linkage between activities of the projects and partly guaranteed increase in
female employment and/or women’s labour force participation.
•
Demonstration of how the priority for entrepreneurship under RC OP has been taken into
account
•
Demonstration on how the project will focus on the targeted group in regards to educational
attainment
The Call for Proposals list three wider priority themes:
•
Enhancing women’s employability
•
Developing practical services facilitating access to labour market and/or to employment
services
•
Raising awareness for tackling obstacles to women’s participation in labour force
The priority themes are supplemented by a non-exhaustive, non-compulsory and indicative list of
actions that the applicants could consider:
•
Development and delivery of career guidance, advisory services including job clubs,
•
Training and retraining in professions/skills that local labour market needs
•
On-the-job training within employers, short term placements within companies i.e. internship
programmes
•
Development and upgrade of technical skills to improve efficiency
•
Activities promoting women to start their own business and including establishing network
•
Determining external finance opportunities for establishing business and supporting
applications to be made to these sources
•
Matching services gathering participants and employers
•
Promoting self-employment and entrepreneurship (consultancy, career guidance)
•
Trainings for motivation and reinforcement for supporting women to access labour market
•
Providing one to one guidance services for women (starting from entrepreneurship training
until setting up the business) who want to set-up a business.
121
•
Developing flexible forms of work and approaches ensuring reconciliation of family and
working life
•
Actions enabling beneficiaries to better access the labour market and/or the employment
services through provision of child and elderly care services and/or other facilities
•
Development of community based child/elderly care services (including training, assessment
needs, model developments)
•
Other pilot services such as community services, customised transport facilities or other
initiatives facilitating access to employment services or to the labour market
•
Field studies related with the above issues including assessment of needs and skills
forecasting, new ways and mechanisms for services financing
•
Awareness and sensitivity raising, information and publicity on tackling cultural obstacles
hindering women’s participation in the labour force
•
Pilot awareness actions at local level (to women, their partners, opinion leaders, employers
etc.)
•
Local partnership approaches including local social dialogue, capacity building for community
development. Effective dialogue between community leaders, employers and those providing services
to women job-seekers
•
Field studies, training tools and methodology developments related with the above issues
122
4. Assessment of local absorption and implementation capacities
Statistical data on local absorption capacities
Table. Number of Applications Selected from Each Region
NUTS II Regions
1 TR A2
Kars
2 TR B2
Van
3 TR C3
Batman
4 TR A1
Erzurum
Diyarbakır
5 TR C2
Şanlıurfa
6 TR C1
Gaziantep
Kayseri
7 TR 72
Sivas
8 TR 90
Trabzon
Elazığ
9 TR B1
Malatya
10 TR 82
Kastamonu
11 TR 83
Samsun
12 TR 63
K.Maraş
Other
provinces
TOTAL
LLL
3
3
0
2
PWE
1
10
4
6
PRE
0
5
0
3
PYE
2
10
4
3
IER
2
17
8
4
Total
8
45
16
18
3
2
4
4
4
1
9
4
12
4
32
15
8
7
22
21
4
2
21
14
10
7
65
51
6
2
4
3
18
3
27
11
1
2
9
1
8
2
22
16
8
4
8
3
41
13
70
34
2
45
4
135
9
41
13
128
2
89
30
438
Table. Number of Applications Not Selected from Each Region
NUTS II Regions
1 TR A2
Kars
2 TR B2
Van
3 TR C3
Batman
4 TR A1
Erzurum
Diyarbakır
5 TR C2
Şanlıurfa
6 TR C1
Gaziantep
Kayseri
7 TR 72
Sivas
8 TR 90
Trabzon
Elazığ
9 TR B1
Malatya
10 TR 82
Kastamonu
11 TR 83
Samsun
12 TR 63
K.Maraş
Other
provinces
TOTAL
LLL
15
74
13
32
PWE
36
166
37
69
PRE
8
40
4
13
PYE
36
167
55
62
IER
40
128
44
37
Total
135
575
153
213
19
24
60
46
14
10
79
66
63
39
235
185
60
47
101
84
15
29
90
81
19
51
285
292
29
20
61
21
81
42
96
47
17
6
11
12
58
53
100
62
31
15
68
19
216
136
336
161
28
443
56
921
18
197
85
994
13
567
200
3122
123
Table. Total Number of Applications Received from Each Region
NUTS II Regions
1 TR A2
Kars
2 TR B2
Van
3 TR C3
Batman
4 TR A1
Erzurum
Diyarbakır
5 TR C2
Şanlıurfa
6 TR C1
Gaziantep
Kayseri
7 TR 72
Sivas
8 TR 90
Trabzon
Elazığ
9 TR B1
Malatya
10 TR 82
Kastamonu
11 TR 83
Samsun
12 TR 63
K.Maraş
Other
provinces
TOTAL
LLL
18
77
13
34
PWE
37
176
41
75
PRE
8
45
4
16
PYE
38
177
59
65
IER
42
145
52
41
Total
143
620
169
231
22
26
64
50
18
11
88
70
75
43
267
200
68
54
123
105
19
31
111
95
29
58
350
343
35
22
65
24
99
45
123
58
18
8
20
13
66
55
122
78
39
19
76
22
257
149
406
195
30
488
60
1056
27
238
98
1122
15
656
230
3560
Table. Percentage of Submitted Applications that Have Been Selected from Each Region
NUTS II Regions
1 TR A2
Kars
2 TR B2
Van
3 TR C3
Batman
4 TR A1
Erzurum
Diyarbakır
5 TR C2
Şanlıurfa
6 TR C1
Gaziantep
Kayseri
7 TR 72
Sivas
8 TR 90
Trabzon
Elazığ
9 TR B1
Malatya
10 TR 82
Kastamonu
11 TR 83
Samsun
12 TR 63
K.Maraş
Other
provinces
AVERAGE
LLL
16,67%
3,90%
0,00%
5,88%
PWE
2,70%
5,68%
9,76%
8,00%
PRE
0,00%
11,11%
0,00%
18,75%
PYE
5,26%
5,65%
6,78%
4,62%
IER
4,76%
11,72%
15,38%
9,76%
Total
5,59%
7,26%
9,47%
7,79%
13,64%
7,69%
6,25%
8,00%
22,22%
9,09%
10,23%
5,71%
16,00%
9,30%
11,99%
7,50%
11,76%
12,96%
17,89%
20,00%
21,05%
6,45%
18,92%
14,74%
34,48%
12,07%
18,57%
14,87%
17,14%
9,09%
6,15%
12,50%
18,18%
6,67%
21,95%
18,97%
5,56%
25,00%
45,00%
7,69%
12,12%
3,64%
18,03%
20,51%
20,51%
21,05%
10,53%
13,64%
15,95%
8,72%
17,24%
17,44%
6,67%
9,54%
6,67%
11,59%
33,33%
15,79%
13,27%
10,73%
13,33% 13,04%
14,81%
12,30%
124
Table. Percentage of Projects Selected from Each Region as Compared to Total Number of Selected
Projects
NUTS II Regions
1 TR A2
2 TR B2
3 TR C3
4 TR A1
5
6
TR C2
TR C1
7
8
TR 72
TR 90
9
10
11
12
TR B1
TR 82
TR 83
TR 63
Kars
Van
Batman
Erzurum
Diyarbakır
Şanlıurfa
Gaziantep
Kayseri
Sivas
Trabzon
Elazığ
Malatya
Kastamonu
Samsun
K.Maraş
Other provinces
TOTAL
Percentage
1,83%
10,27%
3,65%
4,11%
7,31%
3,42%
14,84%
11,64%
9,36%
2,97%
15,98%
7,76%
6,85%
100%
Table. Average Number of Projects Selected per One Million Inhabitants from Each Region
NUTS II Regions
1 TR A2
2 TR B2
3 TR C3
4 TR A1
5
6
TR C2
TR C1
7
8
TR 72
TR 90
9
10
11
12
TR B1
TR 82
TR 83
TR 63
Kars
Van
Batman
Erzurum
Diyarbakır
Şanlıurfa
Gaziantep
Kayseri
Sivas
Trabzon
Elazığ
Malatya
Kastamonu
Samsun
K.Maraş
Other provinces
Number of projects
per million
inhabitants
7
23
9
13
12
7
26
16
23
15
23
13
n/a
125
5. Analysis of the accuracy of the budget allocations under measures
Background information
The funds under IPA component IV are limited and therefore the MIPD focuses a limited number of
priorities that also adapt the ESF regulation's priorities to the particular needs and challenges of
Turkey:
- Attract and retain more people in employment, particularly by increasing labour force
participation of women, and decrease unemployment rates, especially for young people.
- Enhance investment in human capital by increasing the quality of education, improving the
linkage between education and the labour market, and raising enrolment rates at all levels of
education, especially for girls.
- Increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting
lifelong learning and encouraging investment in human resources by enterprises and workers.
- Promote an inclusive labour market with opportunities for disadvantaged people, with a view
to their sustainable integration
Based on these targets, the MIPD proposes the following indicative division of the budget of IPA
component IV in Turkey
Employment
40-50%
Education
30-40%
Social Inclusion
20-25%
The HRD OP elaborates this budget breakdown further and includes funds for Technical Assistance
Employment
44%
Education
30%
Social Inclusion
20%
Technical Assistance
6%
The funds are also divided regionally
National level
20%
NUTS II growth centres
45-55%
NUTS II hinterlands
25-35%
The HRD OP includes the following breakdown of the budget between priorities and measures:
Priority Axis 1
82.150.589
Measure 1.1
27.150.589
Measure 1.2
23.000.000
Measure 1.3
15.000.000
Measure 1.4
17.000.000
126
Priority Axis 2
36.011.764
Measure 2.1
16.011.764
Measure 2.2
20.000.000
Priority Axis 3
20.000.000
Measure 3.1
15.000.000
Measure 3.2
5.000.000
Priority Axis 4
37.341.177
Measure 4.1
30.341.177
Measure 4.2
7.000.000
Priority Axis 5
11.202.355
Measure 5.1
6.202.355
Measure 5.2
3.000.000
Measure 5.3
2.000.000
TOTAL 2007 -2009
186.705.885
127
Annex A5 The Study of Indicators
1. Overview and Introduction: The study on Indicators and the Relevance of the
HRD OP
Part I, II and III of the study are designated to perform an evaluation of the HRD Indicators and is,
according to the TOR, intended to address the following evaluation questions:
Q1.4.3.2 Collect data on indicators not provided through the regular monitoring system
Which evaluation question is repeated as Evaluation Question 3:
Q3 Output and result indicators agreed upon in the HRD OP that cannot be obtained
through monitoring system
Q3.1 Analyse and provide data on output and result indicators agreed upon in the
HRD OP that cannot be obtained through monitoring system
Q1.4.2.4 Analyse the accuracy, clarity, quality, usefulness and internal consistency of the
overall objectives, purposes, targets, indicators and eligible activities at the priority and
measure level of the HRD OP
Q1.4.3.4 Assess the concrete progress of the HRD OP on the basis of the indicators
Q1.3 The quality of projects designed to achieve the objectives, targets and indicators
determined in the HRD OP
The study falls in four Parts.
The first Part describes the fundamental methodology of which parts are previously described in the
Inception Report. Referring to the description of challenges described in the Introduction to the Interim
Evaluation Report, the part further describes the impact of the challenges imposed on the study of
indicators and the limitations to which these challenges impose. The part finally describes the chosen
approach to the study of indicators conditional to the limitations.
Part II makes an evaluation of the Output Indicators, while Part III makes an assessment of the Result
Indicators. Part IV then draws up the main conclusions on the Study of the Indicators of the HRD OP
ending with a summary of the recommendations derived from the study.
PART I: Methodology and approach to the study on indicators
I.1. Methodological note on Evaluation of reform programmes and strategies in Human
Resources
1.1 The rationale for Monitoring and Evaluation
With recent years’ growing public expenditures attention to Monitoring and Evaluation has increased
as a still more important, integral tool in modern management. In management terms Monitoring and
evaluation is of paramount importance, a necessity, primarily due to the combination of scarce
resources and increasing responsibilities. However, and to some extent even more important, the
knowledge economy has added to the importance of the management tools dramatically in three very
interlinked ways:
1.
As information within the information society increases or at least become easier to obtain
and more available (transparent systems) the focus on efficiency with public means has
increased, not least from mass media. Accordingly, the focus on the decision makers’
ability to do an effort has increased;
128
2.
3.
As information becomes highly more accessible the emphasis on knowledge and
knowledge sharing in practical term increases. This necessitates a high demand for
specialist to manage and take care of highly specialist functions at all levels in the
economy. The need of specialists has accentuated the subsequent need for more direct
and sharper measures in Human Resource Development and in that respect the public
sector (including both the educational sector and the Labour Authorities / Ministry of
Labour) often becomes too general. The highly specialised measures are accordingly
often outsourced to specialists within the private sector due to their specialized
competences in e.g. educating people to specialized function or in creating job solutions
and carrier guidance to specialists. Outsourcing to private sector caretakers has obviously
accentuated the need for efficient monitoring;
As global trade markets and financial sector becomes increasingly globalized the
competition between countries and systems become accentuated, mainly due to the
pressure for specialists. In order to at least maintain the current welfare state a sharper
and more committed approach is called for, not least due to increased competition from
transition countries and developing countries. This again accentuates the need for sharper
solutions and accordingly increases the need for efficient measures.
Monitoring and Evaluation are basically two different sides of the same process, since the evaluation
report is supposed to be based on the progress reports which form the feedback from the
implementation phase (i.e. monitoring).
Monitoring is accordingly defined as an on-going process of information collection primarily for the
purpose of programme management. Monitoring focuses on activities and outputs based on an
agreed template for certain agreed periods of time, sometimes even defined as milestones.
Milestones are predefined values of the indicators typically set for each year leading to the end target
value. Accordingly, the set of milestones describes accordingly an expected adjustment path on our
way to meet our established target. For obvious reason we would expect to produce an evaluation
report based on indicators and their measurement against the milestones, allowing us to make
adjustments to our efforts and plan.
Progress reports on the on-going process of implementation might, however, are typically produced on
a more frequent basis in mature boards, councils and ministries.
Evaluation is considered an ex-post assessment of subsequent progress reports in some instances
further explored through targeted intelligence means (e.g. surveys) in order to process the information
collected. Evaluation tends to focus more on outcomes and impacts.
1.2 Methods of Measurement of effect of strategies and programmes
Indicators and target setting is based on measurement and so is the basic fundament for setting up
the complete strategy, see Textbox 1.
129
Textbox 1. Basic principles in all Human Resource Strategies
The planning phase
1. A solid Analysis of the current status
A solid description of the economic conditions which apply at national, regional and
local level by use of Official Statistics and International Indicators;
2. A solid Analysis of current trends and tendencies
A study of the most evident trends and tendencies on the labour market in terms of
new profiles, imbalances and employers’ expectations to the nearest future based
on Labour Market Intelligence elements (Survey, Administrative Records, Partial
Information);
3. A consolidated and confirmed strategy
A consolidated and confirmed strategy based upon clear tasks for all partners
involved, including Social Partners, key employers, the Chambers, Schools and
Universities, local authorities, County Employment Agencies, line Ministries (e.g.
the Ministry of Finance), and similar key stakeholders;
4. Implementation through adequate measures solidly managed and
supervised
Implementation through adequate measures supported by management and
supervision with specific focus on “Value for Money”;
5. Continuous monitoring through agreed milestones
Continuous monitoring through agreed milestones and continuous progress
reports on on-going measures in accordance with the agreed performance
indicators;
6. Evaluation
Evaluation on actions in subsequent Strategies mainly based upon subsequent
progress reports (evaluation sheets, budget, targets, etc.) evaluation of effect and
efficiency (including quality) provides experience and generates scope for new
actions
According to textbox 1 there appear to be six discrete steps in each planning circle. Step 3 and 4 is for
obvious reasons outside the scope for this report on indicators of which step 3 concerns the
establishment of the strategy. The 4 remaining steps, however, all concern the methodology of
measurement.
As all four steps, i.e. step 1, 2, 5 and 6, all are important for the establishment of indicators as well as
the measurement of the impact of our actions, it serves a purpose to describe them in this context.
It serves a purpose, however, to note the words used in step 3: Consolidated and confirmed. No single
agent or body (Ministries, Social Partners, etc.) bears the sole responsibility when it comes to ensure a
balanced and progressive labour market, sustainable economic development, and ensuring a welfare
state. The responsibility is borne by all partners, including single economic agents (schools,
enterprises and individual jobseekers) for obvious reasons.
As soon as the Human Resource Strategy is implemented and actions carried out, feedback from
monitoring the process provides the stakeholders with crucial information about how the market (the
schools, the students, the enterprises, the local authorities, etc.) responds to our interventions. This
information is important as we might need to adjust our actions; i.e. we might need to reinforce actions
as they appear to have less effect than that expected or we might be able to slow down on some of
our action as they appear to have a far stronger impact than expected. Performance indicators and
Targets become important as we will describe in more detail below.
130
The next time we have to update our Human Resource Strategy (e.g. next year) the information
obtained from the adjustment process (called monitoring) should be evaluated as an integral part of
the assessment. The information becomes ‘alive’ since it provides us with information about how
actions work through time and in a local dimension. Please recall here that we never know how
actions will work before they are tested in a real market. Accordingly, our experience becomes
knowledge, which knowledge should be an integral part of the next action plan.
Accordingly, in order to create a knowledge-based updated Human Resource Strategy we need to
make an assessment of the process (evaluation). The logical process of the update can accordingly
be seen as a four-step process with reference to the design of Human Resource Strategies above:
Textbox 2. Basic principles in all Human Resource Strategies
The subsequent evaluation and adjustment phases
A. Assessment of the current situation based on official statistics (Step 1 above)
Key Questions: Where are we? How did we get here? Where are our relative Strengths,
Weaknesses, Opportunities, and Threats?
The study should be based on official statistics only, made comparative at least in terms
of
a. Geography (county level and averages for regional levels and national and
international levels when appropriate);
b. Time; and
c. Other strategic focus areas (e.g. gender, age, economic sectors, etc.).
B. Assessment of the most recent trends (Step 2 above)
Key Questions: How does the market respond? Where are we heading? What are the
potentials, asymmetries, imbalances?
The study should be based on official statistics supported by information and data
obtained through intelligence means (Surveys, administrative records, partial
information, other studies, etc.). As in Step 1 the study should be comparative to ensure
a high degree of transparency.
C. Assessment of current actions (Step 6 above)
Key Questions: What did we do? What were the outcomes (successes and failures)?
How long did it take for the market to respond to our actions (both supply and demand)?
The study should be based on approved performance indicators (re step 3 above) and
made comparative to ensure a high degree of transparency.
D. Proposal for a new confirmed and consolidated strategy
Including:
a. Adjustments based on assessment of steps (A), B and C
b. New actions based on assessment of steps A and B
Accordingly, the mature strategy does not start from scratch, but encompasses an assessment of the
actions in terms of Input, Output, Outcome, and Impact.
131
1.3 The rationale of indicators
The establishment of performance indicators serves at least two general and main purposes:
1.
The indicators are a necessity for continuing Monitoring and Evaluation;
2.
Transparent indicators can be used for benchmarking purposes; International transparent
indicators for international benchmarking, Institutional national indicators can be used for
cross-regional and/or cross-institutional purposes.
The benchmarking element comes in as a tool for strategic knowledge sharing (mainly on institutional
level) or as a platform for strategies. The latter is of course closely related to transparency and
knowledge sharing as it makes good sense to ask “why is this country progressing in a faster pace
than us?”
In that respect and very brief, benchmarking allows us to obtain knowledge of both possible progress
paths and not least knowledge of what works. Practically speaking, we compare our present situation
to our countries facing similar problems and cultural environment which allow us to compete with the
comparable countries on progress. Of course, the competition is done in a friendly environment where
we share our knowledge of successes and failures. That is the whole point in benchmarking, in fact:
knowledge sharing.
In benchmarking indicators plays a crucial role both in our task to find a representative pool of
comparable countries which face the same conditions as our country, and not least in our
benchmarking task to monitor progress.
In monitoring and evaluation as well as in benchmarking the indicators do not answer our basic
questions on “how, why, where, and what”, but they are supposed to be our eyes and ears in our
actions and effort. Small and tiny as they are, they allow us to make a difference.
1.4 Levels of Execution
In order to define indicators in a practical environment it appears natural to distinguish between the
level of performance; e.g. political and institutional contrary to practical implementation, or just central
level contrary to local level. In some specific cases we can identify an additional intermediate level
between in case the objectives, purposes and target groups differ significantly. That could
subsequently be termed multilevel execution of strategies or programmes.
Accordingly, throughout the report we shall implicitly distinguish between different types of processes
which mainly refer to the different purposes of the processes, and accordingly the different types of
evaluations.
The multilevel mechanism does not indicate lack of link between the levels. Rather on the contrary. In
case a multilevel system of indicators and targets is desirable and chosen the links should be
described and justified methodologically.
In case the different levels demands different approach and means in terms of indicators for
measurement of effectiveness slightly different tools in terms of monitoring and evaluation may apply.
However, the basic nature of the content of the levels including the purpose and the objectives and
goals they perceive ought to remain the same.
132
In other words, in some cases it appears necessary to establish different sets of performance
indicators which refer to:
•
First level indicators are meant to measure the overall performance of the overall
programme; i.e. the political / institutional framework
•
Second level indicators are meant to measure the effect of single measures and actions
within the strategy; i.e. practical implementation on local and/ or micro level
This differentiation is even more important during reform processes as the first level of indicators is
supposed to be related to institutional capacity building, adaption of new procedures, rules, and
regimes, build-up of the overall infrastructure, etc. Accordingly, during reform processes the first level
of indicators should reflect (or measure) the impacts of the reform programme itself further to the
single measures and actions undertaken within the new framework. As the institutional build up
becomes more settled and mature we would expect a gradually higher degree of reflection between
the two levels of indicators.
I.e. in the strategy set-up performance indicators should be related to two different levels of the
complete monitoring manoeuvre. Obviously, the two levels are interrelated through the strategy, and
accordingly we would require some connection between the two levels of indicators; i.e. we establish
indicators on the aggregate level which reflects the output / outcome / impact of each single action.
However, to each action several different indicators might apply which do not necessarily have
relevance on the impact of the overall programme / strategy.
As we move on from basic input and basic output to impact the Performance Indicators should reflect
a still higher degree of the basic purposes, goals, and objectives of the complete strategy, the
complete reform, or the overall basic institutional purpose. In that sense as we move from output
through outcome to impact we distinguish between:
1.
The effectiveness of the institutional setup, including accountability and “Value for Money”
2.
The overall national agenda; i.e. in terms of impact in economic and employment terms.
For a complex programme containing multiple operations and projects with each single measure it
accordingly appears logic to investigate the effectiveness of single elements based upon indicators of
input, output and to some extent outcome in terms of “Value for the Money” (input) used to increase
the employability among the target groups.
And then when it comes to the overall programme we identify a set of indicators which measure the
overall relevance and effectiveness of the programme, which mainly include outcome and impact
indicators.
The purpose of using dual systems or multiple levels of performance indicators is obviously to make
the evaluation easier and far more practical since it appears almost impossible to make a solid
assessment of impact in complex programmes or in assessments of a complex national agenda which
easily involves multiple objectives, priorities and actions in multiple dimensions.
1.5 Types of Indicators
In general there are four types of indicators:
1. Input Indicators: Input indicators measure the financial resources provided and the
administrative and regulatory measures taken (for example, resources allocated, resources
used, measures taken, laws passed). The definition of inputs can be treated as very broad
covering in some cases what is often called "process indicators".
133
2. Output Indicators: Output indicators measures the immediate and concrete consequences of
the resources used and measures taken (for example, schools built, teachers employed,
nurses trained) The definition of output covers those goods and services "supplied",
“produced” or "provided" by the public sector with the inputs.
3. Outcome Indicators: Outcome Indicators measures the results at the level of beneficiaries (for
example, gross enrolment rates in primary schools, vaccination). The definition covers the
outcomes (or results) from the use and satisfaction of the goods and services produced by the
public sector - it is where supply comes face-to-face with demand.
4. (Gross) Impact Indicators: Impact Indicators measures the consequences of the outcomes in
terms of wider objectives (for example literacy rates, health improvement). The definition
covers the wider effects of the outcomes but there might also be higher level impacts, related
to broader objectives – growth and income poverty, for example. In order to turn it into a net
impact indicator the effect of other economic, educational, and employment programmes have
to be assessed and accordingly subtracted.
Evaluations based upon these four types of indicators provide a description of the effectiveness of the
programme in terms of impact on the environment and the local society and the overall effect on
employment. But effect and impact are not the only parts within a solid evaluation.
The strategic evaluation of impact has to be closely followed by a financial audit in order to ensure a
sort of “value for money”. In that respect, it is important to understand the need for prioritized
measures and actions.
Accordingly, evaluation has not just to measure an effect in terms of impact but also to evaluate the
financial input to generate the effect; and in case the initial financial input was high, indicating a costly
project, it might be worth reconsider continuing the project – also in case the project appears to
generate a positive impact.
The point is accordingly to measure impact in relation to effort. The consultant has substantial
international experience in evaluating projects, measures, and actions within the Human Resource
sectors of Labour and Employment, within Education, and within Social Affairs.
1.6 Target setting and time frames
1.6.1 The SMART principle
Performance Indicators as well as their targets should be SMART; i.e.
•
S is Specific; indicating that the indicators (as well as the single measure and the single
action) precisely should measure the anchor points in our strategies, measures and
actions;
•
M is Measurable; indicating that the indicators should provide the evaluator and the
executioner with sharp and precise feedback to the actions implemented in a simple,
straightforward and accountable way;
•
A is Affordable or Achievable; indicating that the target as well as the single measure and
the single action, though ambitious and progressive, should be within reach with some
effort;
•
R is Relevant; indicating that the indicator as well as the single measure and the single
action should be directed to create the solutions called for;
134
•
T is Time-bound; indicating that the indicators and targets as well as the single measure
and the single action should follow a predefined and agreed timeframe with clear and logic
milestones and deadlines.
In all matters the SMART principle suggests a pathway to formulating indicators and targets which are
easy to interpret and generally accepted and recognized. As will be clear below it appears less fruitful
to develop a long set of indicators – say, 100 hundred – which generate long report which none are
able to read from an instant, including the analyst/evaluator.
In other words, the indicators should accordingly capture the very essence of the strategy and its main
purpose, and through a sharp and precise list of measurable indicators indicate success or failure from
a general perspective. The precise, concise and short list should facilitate short progress reports which
all are able to read and comprehend.
Further indicators, and especially when we turn to Second Level indicators, can be more distinct and
more specific and detailed, as well as more from a quantitative perspective (more indicators). Again
this aspect only accentuate the focal point that First Level Indicators should encompass or
comprehend Second Level Indicators and/or specialized in-depth indicators as raised above.
The very point is that it detail ensure precise knowledge of the root of the problem, while too much
detail will leave a blurred overall picture. Only if something is terribly wrong and needs to be corrected
we need to turn to the details. Otherwise, we lose focus on the main objectives of the reform process.
Accordingly, the formulating of First Level Indicators becomes important, not just in matters of findings
indicators that can be agreed upon but more so as the First Level Indicators be able to capture the
essence of important variations all operational indicators (Second Level Indicators) without
accentuating single and non-important disturbances in Second Level Indicators.
Finally, the SMART principle accentuates the necessity to establish a timeframe for indicator
measurement (monitoring and evaluation) aligned with targets. Milestones have to be established as
well indicating the expected pathway to achieve the end goal and objectives as described by the
objectives of the strategy. For obvious reason targets should accordingly be assigned to each
milestone within the timeframe not least in order to allow the stakeholder to adjust the actions.
1.6.2 General good practices in Target setting
It is probably impossible to avoid political issues in target setting. It is after all an intrinsically political
process to declare that certain goals will be given prominence and that certain targets will be met by
the incumbent government. However, when all the key issues are represented in a balanced way it
become more difficult for politicians to argue for rapid expansion of access to a system without
accepting the need to commit commensurate resources to maintain quality.
Key principles in target setting are to base targets on operational assumptions, to take local capacities
and resources enough into account when setting targets, to look at past trends in performance as a
basis, drawing upon a sufficiently long time span, and to avoid introducing perverse incentives for
performance. Targets should be set so as to get the balance right between being over-ambitious or
excessively prudent. They should be challenging but achievable. In other words they should be the
product of active engagement by the managers and stakeholders with realities of the system and its
environment.
135
Good practices on these issues include:
•
Clarity of indicators: if indicators are clear and simple it is easier to assess the target
setting objectively.
•
Debate assumptions not forecasts: in other words force the target setting group to think
about the assumptions such as productivity increases, resource availability, which must
occur if the target is to be achieved. Involvement of the Ministry of Finance is important in
that respect.
•
Force them to model those numbers against real data and actual trends.
•
Discuss resources early: Targets depend crucially on resource commitment so resources
need to be added to the equation early.
•
Develop execution skills: Targets are meant to stimulate all those involved in the system
to greater efforts. Targets should be accompanied by clear implementation strategies and
plans which are linked to clear assumptions and ideally show how improved performance
can be achieved at a stable or falling unit cost. Sector managers need to demonstrate
how the targets are to be translated into operational procedures at different levels of the
service or how incentives will be structured to ensure that both public and private sector
service providers move in the desired direction.
Who should be involved in setting targets?
•
Involve all major stakeholders at some point in the process so that the assumptions are
thoroughly debated and so that there is social buy-in from key actors like unions and
private service providers.
•
Give the target setters some independence to increase objectivity but don’t divorce target
setting from the managers who have to deliver the achievement of those targets.
•
Involve a professional statistician in the target-setting group. As illustrated above target
setting is not purely a technical task but it has many technical complexities. Without
professional involvement targets are likely to be formulated without clarity and possibly
without validity.
Target setting should not be seen as a single event occurring once a year or once every few years. It
should be part of an on-going process of operational research around key aspects of systems
performance informed by thorough analysis and regular research.
The basic principle of performance monitoring is that indicators are meant to measure performance
and to provide important input into management decision making and resource allocation. This means
that information from indicators needs to be synchronised with the planning, budgeting and reporting
cycles.
Preliminary targets for the coming year should be proposed by government during the MDA (Ministry,
Department, and Agency) reviews so that what is presented to the annual review involving a wider
stakeholder group is already a well-considered proposal based on tentative resource allocations and a
thorough understanding of the causes of indicator trends.
Ideally the following steps of a well-organised review should be undertaken:
•
Management review: Analysis of performance indicators is presented to responsible
managers in sector ministry for review and comment. Where possible disaggregated data
should be prepared and sent down to the responsible sub-national levels. Managers are
asked to explain significant deviations from targets.
•
Management analysis: Where deviations are major it may be appropriate to analyse the
source of deviation more deeply. (Were resources available as planned? Are there possible
exogenous reasons such as a natural disaster? Is the deviation derived from unexpected
results in certain parts of the country?)
136
•
•
•
•
•
Investigation: It may be appropriate to commission some rapid field assessment to understand
the deviation better. Investigation should also be used to look at factors behind success
stories and develop good practice cases studies for sharing with relevant managers across
the whole system.
Peer Review: Once managers have been allowed an opportunity to analyse, investigate and
understand the performance data there should be an opportunity for them to present this
information and its implications for future action to their peers. Such peer reviews are the best
environment to encourage professional challenge and to ensure that explanations of sector
performance are internally consistent. Such a review should bring in managers from the subnational level. Often they will be in a better position to explain or challenge explanations.
Strategy review: Senior managers of the sector need to absorb this information and develop
initial proposals for strategy change, increased efficiency, resource reallocation, adjustment of
future targets. This would ideally constitute a second forum with more limited membership
following up the work of the one described in the last paragraph.
Inter-ministerial coordination: Most governments are organised in a manner where most
service Ministries cannot implement their strategies without substantial support from other
Ministries such as Finance (financial resource), Public Service (human resources and
government-wide structural reforms), Local Government (coordinator of the local authorities
who actually deliver the services), and Legal Affairs (coordinators of any legislative
programme). So once the sector Ministry has developed a robust analysis of its own
performance and a set of responsive strategies to enhance future performance, it has to go
out and sell its proposals to the rest of government both in Cabinet but also at a more
technical level with its sister Ministries.
Public accountability: The final step in the internal review process is the presentation of
performance data to the appropriate public forum specified by the law or constitution. This
could be a specific Senate committee or an annual parliamentary review of public expenditure.
All of the foregoing should take place before the annual review with the key stakeholders, so that the
annual review can reflect on the domestic debate and enrich it where possible but not become a
substitute for the constitutional organs and channels of debate. The role of the annual review should
then ideally be one of discussion and endorsement.
1.6.3 Progress reports
Progress reports sounds to be a lot more than it actually needs to be, and it appears to impose a lot of
hard work on a continuing basis, both from the side of OBs and the OS. Basically, however, in most
cases the progress reports do not need to be much more than a selection of predefined and agreed
tables (based on the indicators) presumably supplied with some very few words.
The point in progress reports is to provide the members of the council with a management tool. In fact
progress reports are supposed to be a product of a management information system. Normally, if the
progress works as planned, the presentation of progress reports leads to only few comments from the
members. However, in case some of the actions appear to be out of control or if the market
adjustment appears to be lacking, members of the council might call for adjustments to the actions.
In most cases, the council will leave it to the secretariat to make smaller adjustments. Obviously, the
secretariat will then inform the council members on the next coming meeting. However, if more effect
full adjustments are called for the secretariat might need an agreement from the council members
before a new action is taken. In all cases, progress reports constitute the main information channel for
the management (it actually forms an integral part of management information system) and should be
seen as an early warning system.
137
I.2 Approach and limitations to the study of indicators
The basic methodology and approach to the study of the indicators are fairly straightforward and
follows best practices in evaluation of indicators. The outset is based upon an understanding of the
importance of performance indicators, which is lightly described above.
While the above section 8.2 described the purpose and the context of performance indicators to which
we shall refer to in the study of the relevance of the indicators, another main part of the evaluation is to
screen the current selected indicators for quality and availability. In order to understand the study
some comments to evaluation approach is needed, which necessitates a description of the main risks
in establishing indicators. That part will be presented in the context below.
With respect to the approach to evaluate the indicators selected for the HRD OP the following main
steps are needed:
1. Define the context within the selected indicators are placed
2. Organise the indicators in accordance to purpose
3. Obtain data from the monitoring mechanisms in place, and obtain raw data for controlling
purposes as well as to ensure the necessary tools for the evaluation and update of the current
indicators within the system
4. Screen the data based upon a number of factors of validity and control
5. Mirror the data across sections towards control groups (e.g. target groups, regions, etc.,
towards alternate target groups, regions etc.) to assess their relevance in relation to purpose
as well as in order to assess the basic quality of the data
6. Made basic use of the data for analysing purposes, which includes studies of progress etc., in
order to see how well the data responds to changes in environment (including economic
environment not least)
7. Make a qualitative assessment of the data in relation to the objectives, measures, and targets.
For obvious reasons the steps described only reflex the basic overall approach; experience tells us
that multiple problems appear as soon as the evaluation starts which necessitate other, often stronger
means to evaluate the basic quality of the data. Of course that mainly accrues to data and statistics
not provided by official statistical offices since the official statistics has already undergone the basic
validation procedures. Thus dependent on the importance and not least the purpose of the indicators
chosen the basic recommendation is to select well-developed indicators.
The HRD OP operates with two types of indicators only: Output and Result Indicators. Well, that is
basically not entirely true since it only reflex that the HRD OP does not explicitly describe the Input
Indicators. The lack of Input indicators is somehow strange since the HRD OP is using public funding
from EU and the Turkish Government. Nevertheless, the main financial input indicators are described
elsewhere in the HRD OP while it tends to be rather difficult to find the description physical input
indicators. Nonetheless, when it comes to the study of the Input Indicators the Interim Evaluation
refers to the evaluation in a separate chapter above.
The Interim Evaluation then turns the focus on Output and Results Indicators within this chapter.
With respect to the evaluation of the quality of data the Interim Evaluation kindly refer to the limitations
of the study described in section 1.3 in the Introduction, mainly on that part which refers to lack of
basic data. Repeating and accentuating the challenges in obtaining access to the data required and
requested repeatedly by the TAT the above mention 7-steps process of evaluation is seriously
hampered by lack of the raw data; basically the TAT was not provided with any relevant data other
138
than data provided by the OS, TURKSTAT, İŞKUR and the data the TAT could derive from the MIS.
Referring to the above mentioned 7-steps of evaluation the data provided is obviously far from the
anticipated as well as far from enough to make a comprehensive evaluation of the quality of the data.
The TAT was accordingly forced to change strategy and approach. The method used in this study is to
omit step 4 and 5 from the evaluation. In addition, and due to the delays in the overall programme of
the HRD OP also noted in the introduction step 6 can only be done within severe limitations, simply
because the programme cannot be expected to have a real effect so soon after it is commenced.
The limitations impose obviously restrictions to the evaluation as well as its conclusions and
recommendations.
However, the results of the evaluation of the indicators should not be underestimated. In addition to
the technical approach to evaluation of the quality of the indicators, the TAT has launched a
designated survey on the relevance of the Result indicators which in addition to qualitative interviews
with prime stakeholders have added significantly to the outcome of the evaluation.
In particular, the TAT conducted a long half-day meeting with İŞKURstaff in which the relevance of
carefully selected indicators was discussed. Further invitations to undertake the same sessions with
notably the Central Grant Monitoring Teams (CGMTs) were however not doable neither through
follow-up meetings nor through mail correspondence.
This study on indicators is divided into two main parts. The first part in Chapter 9 evaluates the Output
indicators of which the TAT has counted 57 selected indicators from the HRD OP. The second part in
Chapter 10 tries to understand the logic in the 27 selected Result Indicators from a theoretical
perspective in an evaluation of the relevance of the chosen Result Indicators. The importance of the
selection of Result Indicators is thoroughly described.
2.1 The explicit focus on Result Indicators
The stringent focus on Result Indicators in this study, which focus further is accentuated by the
designated survey, needs to be explained.
Turning to the basic definition of output indicators (see section 8.2) the Output indicators measures the
immediate and concrete consequences of the resources used and measures taken. The purpose of
the output indicators is accordingly to measure “how many trainees have participated in a given
project?” and “when we embarked upon the new law framework, did it come through?” In other words
the output indicators specifically address issues of “where did we spend the resources and efforts”
which indicates a very stringent focus on target groups (i.e. those who participated or in more technical
terms; the final beneficiaries).
Output indicators are in general not that difficult to define since they appear almost natural from the
definition of the measures, actions and target groups. Furthermore, output indicators are by nature
almost always of quantitative nature, since the main purpose of output indicators is to measure the
direct effect of the effort; that is numbers.
That means, evaluating the relevance of output indicators soon becomes trivial insofar the output
indicators are closely linked to the target group and measure numbers benefitted. After all, relevance
is a matter of how well the indicator makes sense subject to the measure, the objective and the
priority, and since output indicators almost always make sense, assessing their relevance becomes a
pretty trivial procedure.
139
Result Indicators are of quite different nature. Referring to the section 8.2, we need to define Result
Indicators as a mix of two types of indicators; Outcome Indicators and Impact Indicators.
While the outcome indicators measures the results at the level of beneficiaries (for example, gross
enrolment rates in primary schools, vaccination) and cover the use and satisfaction of the goods and
services produced by the public sector, the Impact Indicators tries to measure the consequences of
the outcomes in terms of wider objectives, and where the difference between output and outcome
describes participants (beneficiaries) of the action in relation to the complete target group including
those who did not participate.
The difference between Outcome and Impact is not arbitrary; while the outcome indicators try to
describe the result for the complete target group in question, the impact indicators focus on the wider
perspective for instance for the economy, the region, the health, etc.
Now, describing the difference between output and result using the mentioned definitions uncover at
least one vital feature: Output measures the programme / project / action undertaken by the
programme itself, while outcome and impact the importance of the programme / project / action from a
broader perspective. It is the difference between local and global. And more so it tends to be the
difference between simple monitoring through progress reports and evaluation of effect.
Further it appears evident that we need to know how our effort and actions, and perhaps not least the
funding, have effect for the conditions of and within the society; it is of paramount importance in a
modern democratic welfare state to make sure that our effort pays off. And in that respect there is a
solid distinct difference in the degree of importance between output and result; we might find it
amusing to invest in some few participants in a given training programme, but the crucial question will
obviously be: What does the investment mean for our society, our economy, our welfare?
And hence the logic in this study’s clear focus on measures which effectively measure outcome and
impact: Perhaps we can live with the fact that our monitoring of the number of participants provides a
slightly incomplete picture of the resources invested, but we cannot live without knowing that what we
do serves a purpose which reaches beyond the limited number of beneficiaries taking part. Obviously,
it is of vital importance to ensure that the selected Result Indicators (to stay in the odd wordings of the
HRD OP) are relevant to the objectives of the effort especially if we, for once, leave our bookkeeper’s
glasses for a moment.
But it does not stop here. In addition to the measuring of the effect to a broader audience, so to speak,
perhaps even including the whole community, as we move from Output via Outcome to Impact, we
achieve to provide some deeper insight to the REAL effect; the effect which lasts (at least longer than
the project period). In other words: we measure sustainability.
And exactly that is the core topic of this study.
140
2.2 The data received from İŞKUR
As noted several times, notably in the Introduction, the TAT has received data from İŞKUR33. The
request for data followed a template in which the TAT noted the data needed which template allowed
the main data providers to insert their comments to the availability of data. The response from İŞKUR
is inserted below:
Table 1 Reply from İŞKUR on request for data on indicators of August 17, 2011
Indicator
Number of registered employed by region (NUTS
2), gender and age
Number of registered unemployed by region (NUTS
2), gender and age
Number of employed through the courses and
counselling, by region (NUTS 2) and gender
Number of entrepreneurs by region (NUTS 2) and
gender
Number of participants entrepreneurship courses by
region (NUTS 2) and gender
Number of women having participated in
entrepreneurship courses for women by region
(NUTS 2)
Number of women having participated in
child/elderly care trainings by region (NUTS 2)
Number of participants in employment oriented
courses by region (NUTS 2) and gender
Number of women having participated in
employment oriented courses designated for
women exclusively
Number of persons having benefited from guidance
and counselling services by region (NUTS 2) and
gender
Rate of young people employed following the
courses, internship and counselling activities by
region (NUTS 2) and gender
Rate of young entrepreneurs by region (NUTS 2)
and gender
Number of young people who participated in
entrepreneurship courses
Number of young people who participated in
employment oriented courses
Number of young people who participated in
internship and apprenticeship programmes
Number of young people having benefited from
guidance and counselling
Number of unemployed people counselled by region
(NUTS 2) and gender
Placement rates of İŞKUR by region (NUTS 2) and
gender
Comments on availability
Registered Number of Employees by age group
does not exist.
available
As a result of drilling through courses and work
placements are available.
Public-private distinction is the number of
establishments.
Years prior to 2009 are not available
Not Available
Not Available
Available
Not Available
Available
As a result of drilling through courses and work
placements are available.
Not Available
Not Available
Not Available
Not Available
Not Available
Guidance and Counselling Services are available
the number of beneficiaries.
Employment Placement on the employment rate
was calculated during the year. Work placements
during the year occurred in previous years an open
circuit is applied to the works of over 100%
placement rate in some cases seem to work. In
addition, Public Works since 2006 by sex
derlenememektedir statistical information in question
was removed.
Besides official statistics supplied from YUIK/Turkstat and data generated from the MIS, İŞKUR was the only
provider of data for the studies in this report. We did not receive data from the other Operation Beneficiaries
despite our continuous requests.
33
141
Indicator
Region (NUTS 2) and sex-specific increase in the
ratio of people who work completed courses in
employability
Percentage of participants who obtained a
certificate in basic skills by region (NUTS 2) and
gender
Number of people who participated in employability
courses provided by networks, by region (NUTS 2)
and gender
Number of training activities on basic skills and
further development of basic skills for the adults
particularly for women
Number of training activities on basic skills for
adults by gender and by region
Number of consultancy services on the
improvement of the adaptability of employers and
employees for social partners, universities, public
and private VET institutions, NGOs, public bodies
and enterprises by region
Increased rate of disadvantaged persons entered
into labour market (having the qualification and
motivation to seek a job) following the rehabilitation
programmes, courses and counselling.
Increased rate of disadvantaged persons employed
following the courses and counselling services
Number of people having participated in guidance
and counselling services
Number of people having benefited from
rehabilitation programmes
Number of people having participated in courses in
scope of ALMPs
Number of people who are reached through the
awareness raising activities
Region (NUTS 2) according to gender and the
central database, monitoring and analysis system
recorded the number of disadvantaged people
Comments on availability
Total job placements Ratio is calculated in the frame
of the course. No information was available before
2007.
No information was available before 2008.
Not Available
See Courses
See Courses
Numerical information is provided by the seminar
activities at workplaces. The information compiled
since 2011 January.
Not Available
Placement into employment rate of disabled people
in the course of total was calculated.
Guidance and Counselling Services are available
the number of beneficiaries.
Guidance and Counselling Services are available
the number of beneficiaries.
Available
?
Available
142
PART II: Output Indicators
II.1 Priority axis 1
To attract and retain more people in employment, particularly by increasing labour force participation
of women, and decrease unemployment rates especially for young people
1.1 Measure 1.1
To promote women’s participation into the labour market, and increase female employment, including
those formerly employed in agriculture
Specific Objectives:
•
To improve employability of women
•
To support women entrepreneurship
•
To diminish cultural and other obstacles that hinder women’s participation in labour
market
Table 1.1 Output indicators on Measure 1.1, Priority axis 1
Count
Indicator
1
Number of women participated
in entrepreneurship courses for
women
2
Number of women participated
in child / elderly care trainings
3
Number of women who
participated in employment
oriented courses for women
4
Number of women benefited
from guidance and counselling
services
Baseline (2006)
2751 women
(1781 people by
İŞKUR and 970
people by
KOSGEB) in
NUTS II regions
N.A. (Data will be
collected when the
measure
implemented)
Total 25813
women (for these
2 indicators there
is just one
integrated
baseline. Women
participated in
employment
guaranteed
courses, guidance
and counselling in
NUTS II region for
3 years)
Target
(2009)
Target
(2011)
Source of Verification
4000
MoLSS Programme
8000 Reports, Final Beneficiary
Reports
1200
MoLSS Programme
2060 Reports, Final Beneficiary
Reports
12000
MoLSS Programme
20600 Reports, Final Beneficiary
Reports
18000
İŞKUR Records, MoLSS
24000 Programme Reports,
Final Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
As mentioned above the output indicators are basically designed to measure progress of the actions.
In that respect the TAT has found and received the following data from the two different sources
available for the TAT as displayed in the following two tables.
143
Table 1.2 Output indicators on Measure 1.1, Priority axis 1 as found in SAR 34
Count
Indicators
HRD OP
1
Number of women
participated in
entrepreneurship
courses for women
2
Number of women
participated in child /
elderly care trainings
3
Number of women who
participated in
employment oriented
courses for women
4
Number of women
benefited from guidance
and counselling services
OIS
Number of women
participated in
entrepreneurship
courses
Number of women
participated in
child/elderly care
training
Number of women
participated in
employment
guaranteed courses
for women
Number of women
benefited from
guidance and
counselling service
Revised: Number of
analyses activities of
the grant scheme
results including the
main lessons learned
Revised: Number of
compendiums
published for
successful granted
projects and results
Baseline
Target
2751 women (
1781 people by
İŞKUR and 970
people by
KOSGEB) in
NUTS II regions
N.A. (Data will be
collected when
the measure
implemented)
Total 25813
women (for these
2 indicators there
is just one
integrated
baseline.
Women
participated in
employment
guaranteed
courses,
guidance and
counselling in
NUTS II region
for 3 years)
0
4000(appr.
2600 in OIS)
0
1
Expected
contribution
2485
PYE: 1803
1200
(approx.
780in OIS)
1030
12000
(approx.
7,800 in
OIS)
1947
18000
(approx.
11,700 in
OIS)
PYE: 3068
2612
PYE: 11864
1
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
Table 1.3 Output indicators on Measure 1.1, Priority axis 1 as found in MIS
Ref.
1
2
3
4
Indicator
Number of women participated in entrepreneurship courses
for women
Number of women participated in child / elderly care
trainings
Number of women who participated in employment
oriented courses for women
Number of women benefited from guidance and
counselling services
Target
Achieved
4.352
2.739
1.686
1.184
3.958
2.209
4.877
2.526
Source: Monitoring Information System, Operating Structure, MoLSS
34
The SAR is dated June 2011 indicating the situation by end 2010. More updated data was received
unfortunately far too late to be incorporated in this study (data of September 2011).
144
1.1.1 Output Indicator no. 1
Table 1.1.1 shows data provided by İŞKUR on Indicator 1.
Table 1.1.1 Output indicators on Indicator 1, Measure 1.1, Priority axis 1 as recorded by İŞKUR:
Number of participants in entrepreneurship courses
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ, Edirne, Kırklareli
Balıkesir, Çanakkale
İzmir
Aydın, Denizli, Muğla
Manisa, Afyon, Kütahya, Uşak
Bursa, Eskişehir, Bilecik
Kocaeli, Sakarya, Düzce, Bolu, Yalova
Ankara
Konya, Karaman
Antalya, Isparta, Burdur
Adana, Mersin
Hatay, Kahramanmaraş, Osmaniye
Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir
Kayseri, Sivas, Yozgat
Zonguldak, Karabük, Bartın
Kastamonu, Çankırı,Sinop
Samsun, Tokat, Çorum, Amasya
Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane
Erzurum, Erzincan, Bayburt
Ağrı, Kars, Iğdır, Ardahan
Malatya, Elazığ, Bingöl, Tunceli
Van, Muş, Bitlis, Hakkari
Gaziantep, Adıyaman, Kilis
Şanlıurfa, Diyarbakır
Mardin, Batman, Şırnak, Siirt
Total TURKEY
2011
(January-July)
MALE
FEMALE
1258
926
206
168
189
224
60
75
834
845
362
316
268
246
504
412
601
545
207
183
315
300
327
245
385
281
235
196
420
334
194
172
133
84
536
429
534
356
143
98
328
186
378
210
883
217
62
63
0
0
54
43
9416
7154
2010
MALE
857
161
62
26
324
115
245
182
66
49
109
178
367
159
197
93
29
423
257
71
167
120
177
119
0
154
4707
FEMALE
657
165
112
24
272
84
246
132
59
51
116
122
192
112
122
57
21
344
211
49
80
103
72
100
0
96
3599
Source: Data provided by İŞKUR
As noted by İŞKUR (see table 1 above) data are only available from 2009 and onwards, which
generally makes the indicators highly questionable, since we have no baseline. Where the baseline of
2751 women (1781 people by İŞKUR and 970 people by KOSGEB) in NUTS II regions as noted in
table 1.1.1 comes from appears to be a mystery if İŞKUR does not have the data. But again, it indicate
that entrepreneurship courses may have been run before 2009, perhaps even before 2006.
Plausible as it may be the point is of course, that without a trustworthy baseline it appears non-sense
to establish a mechanism for monitoring since we cannot investigate progress.
Further, we note that the İŞKUR data shows participants on entrepreneurship courses for both male
and female (and not female alone) which tends to indicate that entrepreneurship courses are not
necessarily designated to women. Table 1.1.2 further elaborates on the data as the table shows the
Female participants in entrepreneurship courses in percentage of total participants in entrepreneurship
courses by region and for total Turkey.
145
Table 1.1.2 Female participants in entrepreneurship courses in percentage of total participants in
entrepreneurship courses by region and for total Turkey, 2010 and 2011 (January to July)
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ, Edirne, Kırklareli
Balıkesir, Çanakkale
İzmir
Aydın, Denizli, Muğla
Manisa, Afyon, Kütahya, Uşak
Bursa, Eskişehir, Bilecik
Kocaeli, Sakarya, Düzce, Bolu, Yalova
Ankara
Konya, Karaman
Antalya, Isparta, Burdur
Adana, Mersin
Hatay, Kahramanmaraş, Osmaniye
Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir
Kayseri, Sivas, Yozgat
Zonguldak, Karabük, Bartın
Kastamonu, Çankırı,Sinop
Samsun, Tokat, Çorum, Amasya
Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane
Erzurum, Erzincan, Bayburt
Ağrı, Kars, Iğdır, Ardahan
Malatya, Elazığ, Bingöl, Tunceli
Van, Muş, Bitlis, Hakkari
Gaziantep, Adıyaman, Kilis
Şanlıurfa, Diyarbakır
Mardin, Batman, Şırnak, Siirt
TURKEY
Female participation
in pct of total
participants by
region
2011
2010
42.4%
43.4%
44.9%
50.6%
54.2%
64.4%
55.6%
48.0%
50.3%
45.6%
46.6%
42.2%
47.9%
50.1%
45.0%
42.0%
47.6%
47.2%
46.9%
51.0%
48.8%
51.6%
42.8%
40.7%
42.2%
34.3%
45.5%
41.3%
44.3%
38.2%
47.0%
38.0%
38.7%
42.0%
44.5%
44.9%
40.0%
45.1%
40.7%
40.8%
36.2%
32.4%
35.7%
46.2%
19.7%
28.9%
50.4%
45.7%
44.3%
38.4%
43.2%
43.3%
Female participation
in pct of total
participants in
Turkey
2011
2010
5.6%
7.9%
1.0%
2.0%
1.4%
1.3%
0.5%
0.3%
5.1%
3.3%
1.9%
1.0%
1.5%
3.0%
2.5%
1.6%
3.3%
0.7%
1.1%
0.6%
1.8%
1.4%
1.5%
1.5%
1.7%
2.3%
1.2%
1.3%
2.0%
1.5%
1.0%
0.7%
0.5%
0.3%
2.6%
4.1%
2.1%
2.5%
0.6%
0.6%
1.1%
1.0%
1.3%
1.2%
1.3%
0.9%
0.4%
1.2%
0.0%
0.0%
0.3%
1.2%
43.2%
43.3%
Source: Own calculation based upon data provided by İŞKUR
Again, we are stunned by doubt; the total number of female participants in total Turkey only accounts
for 43 per cent of total participants. Furthermore, while participants in entrepreneurship course in the
12 NUTS 2 regions participating in the HRD OP only constitute 41.8 per cent of total participants in
these courses, the fraction of women taken part in the courses in the 12 NUTS 2 regions is 40 per cent
(decreasing to 37 per cent in 2011). One can only wonder what the HRD OP was designed to do,
since it appears not clear from looking at the indicator.
Of course, the data from İŞKUR shows all entrepreneurship courses, including those courses which
are run by İŞKUR. In that respect we turn to the Monitoring Information System, which is designed to
follow up on the activities within the programme in table 1.1.2 which clearly states that the HRD OP
has achieved to make entrepreneurship courses for a total of 2.739 women so far (the target be
4.352). Returning then to the data provided by İŞKUR we note that a total of 3691 women have
participated in entrepreneurship courses from January 2010.
The HRD OP should accordingly have achieved to produce a total of 74 per cent of total women in
training courses on entrepreneurship. Does that appear to be sound? Again, the problem is that we
have no data from 2006. And yet, we have; the HRD OP mentions that a baseline in 2006 before the
HRD OP indicates that a total of 2751 women has been trained in 2006 alone, of which 1781 was
trained by İŞKUR.
146
Thus, the facts are:
 The HRD OP has achieved to train 2.739 women
 İŞKUR has recorded that 3691 women has been trained so far in 2010 and 2011
 Accordingly, the HRD OP has achieved to do 74 per cent of all trained women in
entrepreneurship
 However, İŞKUR and KOSGEB, before the HRD OP started, trained 2751 women in 2006
alone (of which İŞKUR trained 1781)
 Assuming that 1781 trained women is the annual average from İŞKUR we would assume that
İŞKUR could train 2671 women in 1.5 years
And the questions to the fact are subsequently:
 Does that imply that due to the HRD OP İŞKUR has achieved to increase the number of
female participants since 2006 by 1020 participants in 1.5 years (or 680 a year)?
 Is it plausible to suggest that the increase in number of female participants in entrepreneurship
courses would never have happened in case HRD OP was never launched?
 Why does the number of participants in the HRD OP selected and HRD OP supported NUTS
2 Regions only constitute 41 per cent of total participants in entrepreneurship courses in
Turkey in 2010-201135?
 Why does the number of female participants in entrepreneurship courses within the 12 NUTS
2 Regions only constitute 40 per cent in 2010 (seeming to decrease to 37 per cent in 2011)
after the implementation of the HRD OP?
The answers to these questions “blow in the wind” for the Interim Evaluation TAT indicating a major
challenge to convince the stakeholders about the effect of the programme.
The basic conclusion is that it is extremely difficult to operate with indicators where the baselines can
be doubted. The Interim Evaluation, however, understands the logic of the output indicator and its
direct link to the purpose (the priority, the measures, and the specific objective (which has not been
subject to further theoretical investigation in this context by the way for the same reason)). Thus, the
Interim Evaluation has no further comments to the indicator per se, though the Interim Evaluation
strongly recommend to find ways and means to make the indicator more forceful and clean through a
strategy of QA in raw data; we simply need to know what we are measuring, and as noted in the,
agreed, critical and sceptical evaluation presented here, the indicators just raise questions which
normally indicates a poor indicator.
Preliminary recommendation36: Clean the indicator through measure of QA in data
collection. Define its focus (what it appear to measure) and create a doable baseline
somewhere which appears convincing (the present baseline is no way convincing as
mentioned).
1.1.2 Output Indicator no. 2
With respect to Output Indicator 2 İŞKUR announces that no data is available (see table 1) which
announcement is repeated in the HRD OP. Accordingly, the indicator cannot be evaluated from a
quantitative perspective.
35
To be fair, the population of the 12 NUTS 2 Regions only constitute 33.6 per cent of total population in Turkey
in 2010. But still we do not know the fraction of participants in training courses in 2006.
36 Throughout this section we use preliminary recommendations only since additional studies of the indicators
from a more overall viewpoint follows below. The Chapter will conclude by stating final recommendations.
147
Nevertheless, it appears slightly difficult to understand the indicator and from where it originates as the
priority, the measures, and the specific objective never address a certain political focus on driving
women into childcare and elderly care. It further sounds strange that the labour markets of the 12
NUTS II Regions should have address a specific or rather exclusive demand for women in childcare
and elderly care; why not men? And by the way, how does that indicator align with the overall
horizontal issue of equal opportunities?
It is not just from a sociological and ethical point of view the indicator sounds strange; from a
theoretical and methodological point it appears very strange that the HRD OP suddenly addresses a
specific sector and links that specific sector to a certain target group. None of the other indicators
address a specific economic sector.
The point is: It might be justified to start projects within childcare and elderly care, and perhaps it is
even just and fair if the majority of participants are women; after all the priority specifically mentions
the gender issue and take note on the employability of women in particular. However, when we
establish indicators, even output indicators, we would prefer to take a certain objective view not
allowing the indicator to be driving force for pushing the actions, priorities, objectives and measures
into a certain direction which otherwise was not intended from an overall perspective. In case we have
a political agenda the political agenda should be clear and explicitly be addressed by the priority, that
is.
Otherwise, the interim evaluation has no further comments at this point; the basics will be subject to
further comments below.
Preliminary recommendation: The Interim Evaluation recommends removing indicators
which appear to contradict the horizontal principles, and tend to distort or create
biases onto the labour market without any reason. It is highly recommended to assure
objective Indicators.
1.1.3 Output Indicator no. 3
Tables 1.1.6-8 shows Participants in employment oriented training courses by gender by region and
for total Turkey for selected years between 2006 and 2010, and 2011 (January to July) in accordance
to data provided by İŞKUR. Table 8.1.1.6 clearly indicates the growth in the number of employment
oriented training courses from 2006 to present.
148
Table 1.1.6 Participants in employment oriented training courses by gender by region and for total Turkey, selected years between 2006 and 2010 and 2011
(January to July)
NUTS
N2
N2
N2
N2
N2
N2
N2
N2
N2
N2
N2
N2
N2
N2
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
N2
N2
N2
N2
N2
TR72
TR81
TR82
TR83
TR90
N2
N2
N2
N2
N2
N2
N2
N0
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ, Edirne, Kırklareli
Balıkesir, Çanakkale
İzmir
Aydın, Denizli, Muğla
Manisa, Afyon, Kütahya, Uşak
Bursa, Eskişehir, Bilecik
Kocaeli, Sakarya, Düzce, Bolu, Yalova
Ankara
Konya, Karaman
Antalya, Isparta, Burdur
Adana, Mersin
Hatay, Kahramanmaraş, Osmaniye
Kırıkkale, Aksaray, Niğde, Nevşehir,
Kırşehir
Kayseri, Sivas, Yozgat
Zonguldak, Karabük, Bartın
Kastamonu, Çankırı,Sinop
Samsun, Tokat, Çorum, Amasya
Trabzon, Ordu, Giresun, Rize, Artvin,
Gümüşhane
Erzurum, Erzincan, Bayburt
Ağrı, Kars, Iğdır, Ardahan
Malatya, Elazığ, Bingöl, Tunceli
Van, Muş, Bitlis, Hakkari
Gaziantep, Adıyaman, Kilis
Şanlıurfa, Diyarbakır
Mardin, Batman, Şırnak, Siirt
TURKEY
2011
(Ocak-Temmuz)
MALE
FEMALE
2426
2209
204
366
214
423
284
479
483
836
456
839
1466
539
968
1147
419
626
188
537
496
512
1654
1449
564
308
432
820
2010
2009
2006
MALE
5031
661
366
1124
1023
422
803
616
532
604
452
1347
571
691
FEMALE
5144
839
472
1008
825
1349
986
791
915
1153
285
537
660
886
MALE
2471
276
110
348
174
503
189
192
453
271
1471
457
388
504
FEMALE
2447
335
100
320
157
699
201
152
534
296
876
216
290
883
354
124
89
645
407
258
161
134
1588
845
121
432
163
1144
548
178
313
185
2279
904
32
37
376
388
72
42
725
634
289
77
337
651
842
427
402
14898
348
74
224
724
796
655
416
17313
590
356
873
359
681
210
450
20170
604
371
479
336
138
191
449
22277
117
82
906
225
460
208
145
10783
153
208
883
254
52
102
194
10825
MALE
FEMALE
52
1
272
29
11
364
0
0
30
25
23
0
348
459
Source: İŞKUR
149
Table 1.1.7 Participation in employment oriented training courses by gender by region and for total Turkey, selected years between 2006 and 2010 and 2011
(January to July): male and female fractions in percentage of total
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ, Edirne, Kırklareli
Balıkesir, Çanakkale
İzmir
Aydın, Denizli, Muğla
Manisa, Afyon, Kütahya, Uşak
Bursa, Eskişehir, Bilecik
Kocaeli, Sakarya, Düzce, Bolu, Yalova
Ankara
Konya, Karaman
Antalya, Isparta, Burdur
Adana, Mersin
Hatay, Kahramanmaraş, Osmaniye
Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir
Kayseri, Sivas, Yozgat
Zonguldak, Karabük, Bartın
Kastamonu, Çankırı,Sinop
Samsun, Tokat, Çorum, Amasya
Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane
Erzurum, Erzincan, Bayburt
Ağrı, Kars, Iğdır, Ardahan
Malatya, Elazığ, Bingöl, Tunceli
Van, Muş, Bitlis, Hakkari
Gaziantep, Adıyaman, Kilis
Şanlıurfa, Diyarbakır
Mardin, Batman, Şırnak, Siirt
TURKEY
2011
(Ocak-Temmuz)
MALE
FEMALE
52.3%
47.7%
35.8%
64.2%
33.6%
66.4%
37.2%
62.8%
36.6%
63.4%
35.2%
64.8%
73.1%
26.9%
45.8%
54.2%
40.1%
59.9%
25.9%
74.1%
49.2%
50.8%
53.3%
46.7%
64.7%
35.3%
34.5%
65.5%
57.8%
42.2%
43.5%
56.5%
39.9%
60.1%
28.9%
71.1%
32.5%
67.5%
45.4%
54.6%
51.0%
49.0%
60.1%
39.9%
47.3%
52.7%
51.4%
48.6%
39.5%
60.5%
49.1%
50.9%
46.3%
53.7%
2010
MALE
49.4%
44.1%
43.7%
52.7%
55.4%
23.8%
44.9%
43.8%
36.8%
34.4%
61.3%
71.5%
46.4%
43.8%
40.5%
58.0%
46.8%
33.4%
37.7%
49.4%
49.0%
64.6%
51.7%
83.2%
52.4%
50.1%
47.5%
2009
FEMALE
50.6%
55.9%
56.3%
47.3%
44.6%
76.2%
55.1%
56.2%
63.2%
65.6%
38.7%
28.5%
53.6%
56.2%
59.5%
42.0%
53.2%
66.6%
62.3%
50.6%
51.0%
35.4%
48.3%
16.8%
47.6%
49.9%
52.5%
MALE
50.2%
45.2%
52.4%
52.1%
52.6%
41.8%
48.5%
55.8%
45.9%
47.8%
62.7%
67.9%
57.2%
36.3%
30.8%
46.8%
34.2%
38.0%
43.3%
28.3%
50.6%
47.0%
89.8%
67.1%
42.8%
49.9%
2006
FEMALE
49.8%
54.8%
47.6%
47.9%
47.4%
58.2%
51.5%
44.2%
54.1%
52.2%
37.3%
32.1%
42.8%
63.7%
69.2%
53.2%
65.8%
62.0%
56.7%
71.7%
49.4%
53.0%
10.2%
32.9%
57.2%
50.1%
MALE
64.2%
8.3%
42.8%
0.0%
0.0%
100.0%
43.1%
FEMALE
35.8%
91.7%
57.2%
100.0%
100.0%
0.0%
56.9%
Source: İŞKUR and own calculations
150
Table 1.1.8 Participation in employment oriented training courses by gender by region and for total Turkey, selected years between 2006 and 2010 and 2011
(January to July): fractions of regional distribution of total training course (pct.)
CODE
TR10
TR21
TR22
TR31
TR32
TR33
TR41
TR42
TR51
TR52
TR61
TR62
TR63
TR71
TR72
TR81
TR82
TR83
TR90
TRA1
TRA2
TRB1
TRB2
TRC1
TRC2
TRC3
TR
GEO
İstanbul
Tekirdağ, Edirne, Kırklareli
Balıkesir, Çanakkale
İzmir
Aydın, Denizli, Muğla
Manisa, Afyon, Kütahya, Uşak
Bursa, Eskişehir, Bilecik
Kocaeli, Sakarya, Düzce, Bolu, Yalova
Ankara
Konya, Karaman
Antalya, Isparta, Burdur
Adana, Mersin
Hatay, Kahramanmaraş, Osmaniye
Kırıkkale, Aksaray, Niğde, Nevşehir, Kırşehir
Kayseri, Sivas, Yozgat
Zonguldak, Karabük, Bartın
Kastamonu, Çankırı,Sinop
Samsun, Tokat, Çorum, Amasya
Trabzon, Ordu, Giresun, Rize, Artvin, Gümüşhane
Erzurum, Erzincan, Bayburt
Ağrı, Kars, Iğdır, Ardahan
Malatya, Elazığ, Bingöl, Tunceli
Van, Muş, Bitlis, Hakkari
Gaziantep, Adıyaman, Kilis
Şanlıurfa, Diyarbakır
Mardin, Batman, Şırnak, Siirt
TURKEY
2011
(Ocak-Temmuz)
MALE
FEMALE
16.3%
12.8%
1.4%
2.1%
1.4%
2.4%
1.9%
2.8%
3.2%
4.8%
3.1%
4.8%
9.8%
3.1%
6.5%
6.6%
2.8%
3.6%
1.3%
3.1%
3.3%
3.0%
11.1%
8.4%
3.8%
1.8%
2.9%
4.7%
2.4%
1.5%
0.8%
0.9%
0.6%
0.8%
4.3%
9.2%
2.7%
4.9%
1.9%
2.0%
0.5%
0.4%
2.3%
1.3%
4.4%
4.2%
5.7%
4.6%
2.9%
3.8%
2.7%
2.4%
100.0%
100.0%
2010
MALE
24.9%
3.3%
1.8%
5.6%
5.1%
2.1%
4.0%
3.1%
2.6%
3.0%
2.2%
6.7%
2.8%
3.4%
0.6%
2.1%
0.8%
5.7%
2.7%
2.9%
1.8%
4.3%
1.8%
3.4%
1.0%
2.2%
100.0%
2009
FEMALE
23.1%
3.8%
2.1%
4.5%
3.7%
6.1%
4.4%
3.6%
4.1%
5.2%
1.3%
2.4%
3.0%
4.0%
0.8%
1.4%
0.8%
10.2%
4.1%
2.7%
1.7%
2.2%
1.5%
0.6%
0.9%
2.0%
100.0%
MALE
22.9%
2.6%
1.0%
3.2%
1.6%
4.7%
1.8%
1.8%
4.2%
2.5%
13.6%
4.2%
3.6%
4.7%
0.0%
0.3%
0.3%
3.5%
3.6%
1.1%
0.8%
8.4%
2.1%
4.3%
1.9%
1.3%
100.0%
2006
FEMALE
22.6%
3.1%
0.9%
3.0%
1.5%
6.5%
1.9%
1.4%
4.9%
2.7%
8.1%
2.0%
2.7%
8.2%
0.0%
0.7%
0.4%
6.7%
5.9%
1.4%
1.9%
8.2%
2.3%
0.5%
0.9%
1.8%
100.0%
MALE
0.0%
0.0%
0.0%
0.0%
14.9%
0.3%
78.2%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
6.6%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
100.0%
FEMALE
0.0%
0.0%
0.0%
0.0%
6.3%
2.4%
79.3%
0.0%
0.0%
0.0%
6.5%
5.4%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
0.0%
100.0%
Source: İŞKUR and own calculations
151
Table 1.1.9 Share and fraction on Participation in employment oriented training courses by gender by
region and for total Turkey, between 2006 and 2010 and 2011 (January to July): Share of total
participants in HRD OP regions (total and female), fraction of female in HRD OP Regions, Share of
total participants in regions not taking part in HRD OP and the fraction of female participants in these
non-HRD OP Regions (pct.)
2006
2007
2008
2009
2010
2011
SHARE
Total
TOTAL TR
2.9%
9.1%
36.4%
31.8%
30.2%
35.6%
SHARE
HRD/total
Female
0.0%
15.7%
43.9%
32.7%
30.4%
36.8%
FRACTION
Fem/total
HRD OP R
0.0%
50.8%
44.9%
51.5%
52.8%
55.6%
SHARE
NHRD/total
Female
100.0%
84.3%
56.1%
67.3%
69.6%
63.2%
FRACTION
Fem/total
Non-HRD R
58.5%
27.3%
32.9%
49.4%
52.4%
52.7%
Source: İŞKUR and own calculations
Table 1.1.9 summarizes the study which can be derived from the three tables before. The first column
shows the share of reported participants in employment oriented training courses from HRD OP NUTS
2 regions relative to total Turkey. The second column shows the same for all female participants. The
third column presents the fraction of female participated in employment oriented training courses from
HRD OP NUTS 2 regions in percentage of total participants (same regions, of course). The two last
columns repeat the same as the two middle columns; just this time for regions not taking part in the
HRD OP.
The table provide reason to do some thoughts. We have already noted the sharp increase in the
number of participants in employment oriented training courses, but somehow the fraction of
participants does not really show to be directed to the regions taking part in the HRD OP. The share of
total participants in all employment oriented training courses of Turkey appears to be in the
neighbourhood of 30-35 per cent.
If we look at the number of female participants of HRD OP oriented NUTS 2 regions relative to total
female participants in employment oriented training courses there appear to be no difference. The
exception is 2008 which is before the start of the HRD OP projects. Accordingly, it appears slightly
difficult to see the direct effect of the HRD OP, which notably focuses not just on female participation
in employment oriented training courses, but also tries to stimulate employment oriented training
courses specially designated for women. If the HRD OP should reinforced employment oriented
training courses specially designated for women we would have expected to see a higher fraction of
female participants in HRD OP oriented regions.
The only real positive direct effect is seen in column 3 compared to column 5; it appears that the
fraction of females taking part in employment oriented training courses are slightly higher in HRD OP
oriented regions. However, that effect stops by the commencement of the HRD OP.
These observations served a purpose, and the purpose was not the observations derived from
studying the tables. The point was to show how easy it is to make good analysis when presented for
reliable data as well as an indicative indicator. Of course we could question the effect of the HRD OP
on this basis, but first of all the data was not based upon HRD OP induced projects but both HRD OP
projects and initiatives originated by İŞKUR, and furthermore we have to admit that the HRD OP
needs time to provide effect, including the direct effect which output indicators are designed to
measure.
152
Accordingly, the conclusion is that the indicator, provided that the monitoring unit finds ways to ensure
similarly solid data as provided by İŞKUR in this case, is an effective indicator for monitoring purposes.
Again, however, relating the data from İŞKUR with the data from MIS presented in table 1.1.2 creates
multiple questions which we do not need to repeat here. And true, the data from İŞKUR do NOT just
show Number of women having participated in employment oriented training courses for women, but
Number of women having participated in employment oriented training courses for both men and
women. However, and still, the data provided by İŞKUR are useful, indicating that in case the
Operating Structure with help from İŞKUR finds ways to clean the data from 2006 and onwards, a
solid indicator is created.
Preliminary recommendation: The Interim Evaluation recommends keeping the
indicator on female participation on employment oriented training courses provided
that the data are validated, consistent, and based upon solid QA measures of control.
1.1.4 Output Indicator no. 4
Output indicator no. 4 concerns “Number of women benefited from guidance and counselling
services“. Again data is provided by İŞKUR which institution is the true bearer of the responsibility of
providing documentation. The study of the indicator is based upon the same technique as on indicator
3 and the conclusion from indicator 3 remains the same. Accordingly, the Interim Evaluation Report
finds no reason to repeat the long documentation on this indicator as well 37.
Preliminary recommendation: The Interim Evaluation recommends keeping the
indicator on Number of women benefited from guidance and counselling services
provided that the data are validated, consistent, and based upon solid QA measures of
control. Again the additional would be to ensure data on both genders as well as obtain
data from other Turkish regions in order to ensure control groups.
1.2 Measure 1.2
To increase youth employment
Specific Objective:
•
To promote integration of young people into the labour market
Table 1.2.1 Output indicators on Measure 1.2, Priority axis 1
Count
5
6
7
Indicator
Baseline (2006)
Number of young people who
970 people in
participated in entrepreneurship
NUTS II regions
courses
Number of young people who
participated in employment
oriented courses
Number of young people who
participated in internship and
apprenticeship programs
Total 8562 young
people (for these 3
indicators there is
just one integrated
baseline. young
people participated
Target
(2009)
1500
6000
9000
Target
(2011)
Source of Verification
MoLSS Programme
2500 Reports, Final
Beneficiary Reports
MoLSS Programme
10000 Reports, Final
Beneficiary Reports
MoLSS Programme
12000 Reports, Final
Beneficiary Reports
37
Basically, it is not the job of the Interim Evaluation Report to make the progress reports. In this case, and not
least due to save both time and paper, the Interim Evaluation Report refers to the techniques used in sub-section
8.2.1.3. The data obtained from İŞKUR should already be in the hands of the Monitoring and Evaluation Unit of
MoLSS; otherwise the author of this report will happily submit the data in electronic form.
153
Count
8
Indicator
Target
(2009)
Baseline (2006)
in employment
guaranteed
Number of young people having
courses, internship
benefited from guidance and
programmes and
counselling
guidance and
counselling)
Target
(2011)
9000
Source of Verification
MoLSS Programme
12000 Reports, Final
Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
Table 1.2.2 Output indicators on Measure 1.2, Priority axis 1 as found in SAR
Count
5
6
7
8
Indicators
HRD OP
OIS
Number of young people
Number of young
who participated in
people who
entrepreneurship courses participated in
entrepreneurship
courses
Number of young people
Number of young
who participated in
people who
employment guaranteed
participated in
courses
internship,
employment
Number of young people
guaranteed
who participated in
courses and
internship and
apprenticeship
apprenticeship programs
programs
Number of young people
Number of young
having benefited from
people having
guidance and counselling benefited from
guidance and
counselling
Baseline
Target
970 people in
NUTS II regions
At least 1500
Expected
contribution
1803
PWE: 2489
Total 8562
young people
(for these 3
indicators there
is just one
integrated
baseline, young
people
participated in
employment
guaranteed
courses,
internship
programmes and
guidance and
counseling)
6000
3068
PWE: 1947
9000
2410
9000
11864
PWE: 2612
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational
Programme, 2010, Operating Structure, MoLSS, dated June 2011
Table 1.2.3 Output indicators on Measure 1.2, Priority axis 1 as found in MIS
Ref.
5
6
7
8
Indicator
Number of young people who participated
entrepreneurship courses
Number of young people who participated
employment guaranteed courses
Number of young people who participated
internship and apprenticeship programs
Number of young people having benefited
guidance and counselling
Target
in
in
in
from
Achieved
4.944
2.311
5.950
2.310
5.023
1.426
17.130
2.480
Source: Monitoring Information System, Operating Structure, MoLSS
1.2.1 Output Indicator no. 5
The first output indicator under Priority Axis 1, specific objective 2 is Number of young people who
participated in entrepreneurship courses. As noted above no data are available for this indicator.
Nonetheless, the HRD OP stipulates a target of 1500 in 2009 increasing to 2500 in 2011 from a
baseline of 900 in 2006. Where the HRD OP is able to obtain the data from remains a mystery 38.
38
In general the HRD OP never names its source which is a disqualifying feature in all public reports. Similarly the
use of self-invented acronyms without a glossary list is very common in the supporting documents, which does
not make the documents easier to read; sometimes the reader simply has to make an attempt to guess.
154
The MIS on the other hand announces a target of 4944 and stipulates that so far 2311 young people
has benefitted from entrepreneurship courses. We have to assume that the data from MIS is based
upon data reported directly from the participating projects.
Again something appears slightly odd; while the HRD OP announces a target of 2500 in 2011 the SAR
reports a target of at least 1500 and yet the MIS announce that so far 2311 has participated under the
programme in the pursuit of a target of 4944. Somehow these figures do not match and it tends to be
difficult to see what the HRD OP actually is heading for.
Preliminary recommendation: There is basically no theoretical or logical objection
towards the indicator insofar consistent data is available. It is highly recommended to
find ways to agree on the target (as well as the achievements). An additional note could
be that the indicator clearly matches indicator 1 under specific objective 1 studied
above; it makes sense to have indicators which are fruitful for multiple purposes.
Further, it would again be recommendable to ensure valuable control groups to see the
net effect of the measure.
1.2.2 Output Indicator no. 6, 7, and 8
Again, we have no raw data on:
 Number of young people who participated in employment guaranteed courses (indicator no. 6)
 Number of young people who participated in internship and apprenticeship programs
(indicator no. 7)
 Number of young people having benefited from guidance and counselling (indicator no. 8)
With respect to targets and baselines the documents appear to agree with each other and with MIS.
However, it is generally not recommendable to use complex or aggregated measures as baselines;
here the baseline aggregates the three named indicators.
The point is that aggregated baselines make it impossible to see progress since the commencement
of the HRD OP, which basically disqualifies the baseline since it serves no point.
Another point from a practically perspective is of course, that it appears to fairly difficult to separate the
three actions from each other in the process; normally we would invite potential participants to
employment guaranteed courses or, and notably, internship and apprenticeship programs, into an
interview before we assign them to the measures as part of the visitation process. And often that
interview has close links to what we would define as “guidance and counselling”, since that is simply
what it is when we try to persuade clients into something which they hopefully would benefit from. And
sometimes it could even easily be difficult to distinguish clearly between employment guaranteed
courses and internship and apprenticeship programmes; most times both are present in the same
action.
Accordingly, we cannot even be sure that the same participants are not counted twice; at least it would
be difficult for İŞKUR to separate the three in their daily work, notably if they experience drop-out from
one of the two former actions39. But more so, it is generally not particular fair and just to separate the
counting into three brackets since they are mutually related to an obvious extent.
The best guess of the meaning of PWE as mentioned in table 1.2.3 is “Promoting Women’s Employment” as far
as the TAT can make it.
39 Please note here, that we assume that the Monitoring and Evaluation Unit understand the importance to have
control groups in order to investigate the net effect of the HRD OP. What İŞKUR do is mainly addressed to the
control group. Well, basically it is the same for the project beneficiaries, since they experience the same
difficulties in separating between the three where at least the third appears to form a natural part of the two
former. And yet still, all effort has to be reported upon, which notably becomes an issue in case of drop-outs.
155
Preliminary recommendations:
 There is basically no theoretical or logical objection towards the indicators
insofar consistent data is available and the indicators can be tracked and
recorded practically and with ease for the stakeholders
 However, the close relation between the three indicators can never be avoided,
and it is doubtful that conflicts and difficulties in double counting can be
completely eliminated
 Therefore we recommend to do the following:
o Aggregate the indicators into one indicator but keep the three indicators
on an internal operational level, providing strict guidelines to coding
and quality assurance (QA)
o Eliminate the third indicator on guidance when related to the other two
indicators but keep the indicator as measuring separate actions only
related to guidance and counselling (that would presumably necessitate
a far lower target)
o Ensure solid guidelines to separate the two former actions from each
other, and make explicit notes to what to be reported and measured in
case of drop-outs
1.3 Measure 1.3
To promote registered employment
Specific Objective:
•
This measure aims to promote registered employment through strengthening the capacity
of social security institution, provision of awareness raising and ensuring coordination
among relevant public institutions and social partners.
Table 1.3.1 Output indicators on Measure 1.3, Priority axis 1
Target
(2011)
Indicator
Baseline (2006)
Target
(2009)
9
Number of established local
partnerships
N.A. (There was
no such holistic
partnership
approach before)
15
(in Growth
Centres)
MoLSS Programme
30 Reports, Final
Beneficiary Reports
10
Number of SSI staff at central
and local level who participated
in trainings
827
1200
MoLSS Programme
2400 Reports, Final
Beneficiary Reports
11
Number of staff from social
partners who participated in
trainings on National Strategy
N.A. (The
strategy is a new
one)
600
MoLSS Programme
1000 Reports, Final
Beneficiary Reports
Count
Source of Verification
Source: Human Resource Development Operational Programme, second version
Table 1.3.2 Output indicators on Measure 1.3, Priority axis 1 as found in SAR
Count
Indicators
HRD OP
9
Number of established
local partnerships
10
Number of SSI staff at
central and local level
who participated in
OIS
Number of
permanent local
partnerships to
promote registered
work established
and operating
Revised: Number
of SSI, relevant
institutions' staff and
Baseline
N.A. (There
was no such
holistic
partnership
approach
before)
827
Target
15 (in Growth
Centres)
1200
Expected
contribution
336
94
156
trainings
11
Number of staff from
social partners who
participated in trainings
on National Strategy
social partners who
participated in
trainings to promote
registered
employment
improved.
N.A. (The
strategy is a
new one)
600
673
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational
Programme, 2010, Operating Structure, MoLSS, dated June 2011
Table 1.3.3 Output indicators on Measure 1.3, Priority axis 1 as found in MIS
Ref.
9
10
11
Indicator
Target
Number of established local partnerships
Number of SSI staff at central and local level who
participated in trainings
Number of staff from social partners who participated in
trainings on National Strategy
Achieved
379
163
3.228
151
4.122
1.125
Source: Monitoring Information System, Operating Structure, MoLSS
Interim Evaluation’ comments to Output Indicators underneath Measure 1.3
The TAT has received no raw background data and is accordingly not able to make an assessment of
the quality of the data used in the tables above.
The main comment to the three indicators must be that the targets announced in MIS do not align with
the target of the HRD OP and the SAR. The Interim Evaluation has no further comment to that
observation.
With respect to the relevance of the indicators the Interim Evaluation has found no serious objections
insofar the indicators make sense for the stakeholders.
1.4 Measure 1.4
To improve public employment services
Specific Objective:
•
To improve the quality and efficiency of public employment services especially by
strengthening administrative capacity.
Table 1.4.1 Output indicators on Measure 1.4, Priority axis 1
Count
Indicator
Baseline (2006)
12
Number of İŞKUR staff who
participated in PES trainings
301
13
Number of Monitoring
Mechanisms
0
Target
(2009)
Target
(2011)
1000
1
Source of Verification
MoLSS Programme
1200 Reports, Final
Beneficiary Reports
The mechanism itself,
MoLSS Programme
1
Reports, Final
Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
157
Table 1.4.2 Output indicators on Measure 1.4, Priority axis 1 as found in SAR
Count
Indicators
HRD OP
12
13
14
15
16
17
18
Baseline
Target
OIS
301
1000
0
1
Number of İŞKUR
staff who
participated in PES
trainings
Number of
Monitoring
Mechanisms
Number of monitoring
mechanisms
established for ensuring
the sustainability of
monitoring of ALMP
through cooperation
with Provincial
Employment and
Vocational Training
Boards.
Number of İŞKUR
provincial offices
transformed into model
offices by being
modernised and
becoming user-friendly
to provide customerfocused public
employment services
Number of İŞKUR staff
in newly established
model offices that
increased their
knowledge/abilities on
job search methods,
customer relations and
managerial skills
Number of İŞKUR staff
that increased their
knowledge/abilities on
conducting labour
market analyses
Number of İŞKUR and
MoLSS staff at central
level responsible for IT
related issues that
increased their
knowledge/ability on
Software development
training, Security
Training, Database
Training, and Network
Training
Divided into two: 1)
Increase in the
percentage of qualified
software relating to
labour market
information system of
İŞKUR
Expected
contribution
Data on the
indicators has
not been
provided in the
progress report;
it was noted as
an information
that shall be
submitted to the
OS.
23
450
160
50
158
Count
19
20
21
22
23
24
25
26
Indicators
2) Increase in the
percentage of the
usage of software
quality standard
Revised: Increase in
the percentage of
qualified Internet based
services of İŞKUR
Number of İŞKUR staff
at provincial levels that
increased their
knowledge/abilities on
job and vocational
guidance services
Result in the OP,
Output in the OIS:
Number of the
unemployed people
benefited from job and
vocational guidance
services and increased
their knowledge on job
and career perspectives
Number of İŞKUR staff
responsible for
designing the Turkish
Occupational Dictionary
that increased their
knowledge/abilities on
ISCO 2008 to adopt
Turkish Occupational
Dictionary
Number of İŞKUR and
MoLSS staffs at
provincial levels that
increased their
knowledge/abilities on
monitoring ALMP
Revised: Increase in
the percentage of
effective of ALMP
measures determined
in the 15 growth
centres.
Number of the
members of Provincial
Employment and
Vocational Training
Boards and the experts
working in cooperation
with these Boards who
increased their
knowledge/ability
regarding employment
issues
Baseline
Target
Expected
contribution
0%
200
12
18654
22000
(Male: 11000;
Female:
11000)
10
500
430 members
and experts
159
Count
27
28
29
30
31
Indicators
Number of staff of
MoLSS, İŞKUR and
representativeness of
social actors who
increased their
knowledge on
European Employment
Strategy, employment
policies of the Member
States (best practices),
urgent employment
policy measures in case
of global crisis and the
Member States
practises in designing
employment policies
Number of research
centres established in
the library at the
disposal of MoLSS to
assist the staff
designing employment
policies.
Number of models
designed cooperation
and coordination
between MoLSS and
Ministry of Science,
Industry and
Technology is
increased regarding
designing employment
policies.
Number of call centres
(7/24 functioning)
established
Number of İŞKUR staff
who gained the ability
and knowledge on
working in a call centre
and informing people in
an efficient way
Baseline
Target
Trainings: 100
people in total;
Internships: 30
people in total;
20 from
İŞKUR, 5 from
MoLSS and 5
from social
partners
0
1
0
1 model
through a
policy paper
0
1
NA (Data will be
collected when
the measure
implemented)
125
Expected
contribution
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
1.4.1 Interim Evaluation’ comments to Output Indicators underneath Measure 1.4
The TAT has found no reliable data on the indicators in the MIS, and again no raw data has been
submitted to the TAT.
The OIS, which is defined as a so-called Operating Identification Sheet, has transformed the indicators
of the programme, and not to the easier route; the 2 output indicators of the HRD OP suddenly appear
to be 17 where one of the additional 15 was a former result indicator.
The result indicator which now is an output indicator is: “Number of the unemployed people benefited
from job and vocational guidance services and increased their knowledge on job and career
perspectives”. The transformation of the single result indicator to an output indicator is correct;
basically it is an output indicator. The reason which is methodological is given within the context in the
section on Result Indicators below as well as in the methodological annex; the target group only
concerns those who receive the benefit from the programme.
160
Otherwise, the general point on behalf of the Interim Evaluation is that the OIS describes the content
of the indicators. For instance, “Number of Monitoring Mechanisms” (Indicator no. 13 above) is not
sharp enough, since we simply do not know what the content is. In this case, the OIS redefined the
indicator to: “Number of monitoring mechanisms established for ensuring the sustainability of
monitoring of ALMP through cooperation with Provincial Employment and Vocational Training Boards”.
Obviously, far more precise.
For some reason the OIS does not specify the meaning of Output Indicator 12 “Number of İŞKUR staff
who participated in PES trainings”. However, exactly that appear divided into several new indicators
within the OIS, which clearly tells us the content of the trainings. And that appears to be the main point
in the additional output indicators; they all appear to specify the content and the exact measure of the
indicator.
Though it is not possible to evaluate the quality of the data in this section, there appear no obvious
practical hindrances to obtaining data from a theoretical point of view; all indicators are well-defined
and sharp, and it should be fairly easy to create routes for the feedback data needed.
The Interim Evaluation has no further comments to the indicators from a qualitative, theoretical point of
view insofar the additional indicators are purely serve internal purposes. If not, the Interim Evaluation
has to issue a warning, which will be further described below: There are already far too many
indicators within the HRD OP40. The section below presents a solution to the obvious difficulties in
decreasing the number of indicators, which solution is fairly similar to the model mentioned here.
II.2 Priority axis 2
To enhance investment in human capital by increasing the quality of education, improving the
linkage between education and labour market, and raising enrolment rates at all levels of
education, especially for girls
2.1 Measure 2.1
To increase enrolment rates particularly for girls at all levels of education and vocational
training
Specific Objectives:
•
To increase enrolment rates particularly for girls in secondary education and VET
•
To decrease drop-outs particularly for girls
•
To increase vocational skills and competences of labour force, particularly of girls and
women
•
To promote awareness-raising for parents on importance of education, especially for girls.
40
This note was already mentioned by the ex-ante evaluation.
161
Table 2.1.1 Output indicators on Measure 2.1
Count
Target
(2009)
1720
Target (2011)
+ 10750
18750
Attendance Sheets,,
MoLSS Programme
Reports, Final
Beneficiary Reports
2150
3550
NA (There was
no such holistic
protocols)
43
53
Participants List,
MoLSS Programme
Reports, Final
Beneficiary Reports
Protocols signed
between local
administrations
enterprises schools
NGOs and social
partners
NA (It will be
determined after
the measure is
implemented)
0
2000
7000
43
43
77 %
80-85%
85%
Indicator
Baseline (2006)
32
Number of educational
staff having participated
in vocational guidance
services trainings
33
Number of parents who
are given trainings
34
Number of NGO members
having participated in
trainings
0 (There was no
such a collective
training except
some individual
ones)
8248 parents
have been
reached between
2001-2006 (in
Adıyaman,
Bingol,
Diyarbakır,
Erzurum,
Şanlıurfa, Hatay)
NA
35
Number of protocols
between local
administrations
enterprises schools NGOs
and social partners in
order to increase girls’
enrolment rates
Number of families
individually visited to
persuade to send their
daughters to school
Action plans prepared
36
37
38
Secondary vocational
schools equipped
2920
Source of
Verification
Participant List
MoLSS Programme
Reports, Final
Beneficiary Reports
Attendance sheets,
MoLSS Programme
Reports, Final
Beneficiary Reports
MoLSS Programme
Reports, Final
Beneficiary Reports
School enrolment
rates
MoNE statistics,
MoLSS Programme
Reports, Final
Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
Table 2.1.2 Output indicators on Measure 2.1 as found in SAR
Count
32
33
34
Indicators
HRD OP
OIS
Number of
Revised: Number of
educational staff
teachers participated
participated in
in trainings about
vocational guidance vocational guidance
services training
services at schools
Number of parents
Revised: Number of
who are given
parents trained by
trainings
programmes aiming
at raising the
awareness of
parents on the
importance of
education
Number of NGO
members
participated in
trainings
Revised:Number of
NGO members
participated in
trainings aiming at
raising the
Baseline
Target
0 (There was no
such a collective
training except
some individual
ones)
8248 parents
have been
reached between
2001-2006 (in
Adıyaman,
Bingol,
Diyarbakır,
Erzurum,
Şanlıurfa, Hatay)
NA
1720 (40 teachers
from 43
provinces)
Expected
contribution
23
plus 10750
4001
2150
320
162
Count
35
36
39
40
41
42
Indicators
awareness of
parents on the
importance of
education
Number of protocols Revised: Number of
between local
protocols signed
administrations,
between public
enterprises,
institutions, social
schools, NGOs and
partners,
social partners in
municipalities,
order to increase
universities and
girls’ enrolment
NGOs aiming at
rates
increasing
cooperated activities
for supporting
education of girls
Number of families
Revised: Number of
individually visited
parents families
to persuade to send reached through
their daughters to
campaigns aiming at
school
raising their
awareness on the
importance of
education for
persuading them to
send their daughters
to school
Number of pilot
projects
implemented to
increase the quality
of VET education
and girls’ schooling
and quality
education
Revised: Number of
VET programmes
prepared according
to the regional
needs with the
cooperation of all
partners
Revised: Number of
schools (secondary
education
institutions
especially those for
girls) equipped in
the pilot provinces
to contribute to girls’
schooling and
quality education
Number of detection
and monitoring
services for students
particularly for girls
under the risk of
dropping out or who
have already
dropped out in
primary and
secondary schools
Baseline
Target
Expected
contribution
NA (There was
no such holistic
protocols)
43
24
NA (It will be
determined after
the measure is
implemented)
20000
5639
App.(-) 15000
0
Approximately 5080 projects
X
NA (It will be
determined after
the measure is
implemented)
Not accessible
data yet
0
(not exceeds 15%
of total budget)
Not accessible
data yet
0
1
Not accessible
data yet
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
163
Table 2.1.3 Output indicators on Measure 2.1 as found in MIS
Ref.
32
33
34
35
36
37
38
Indicator
Target
Number of educational staff having participated in
vocational guidance services trainings
Number of parents who are given trainings
Number of NGO members having participated in trainings
Number of protocols between local administrations
enterprises schools NGOs and social partners in order to
increase girls’ enrolment rates
Number of families individually visited to persuade to send
their daughters to school
Action plans prepared
Secondary vocational schools equipped
Achieved
2089
274
18164
545
481
4687
56
451
12698
2955
Source: Monitoring Information System, Operating Structure, MoLSS
2.1.1 Interim Evaluation’ comments to Output Indicators underneath Measure 2.1
Again, the Interim Evaluation had no access to raw data.
In general and as seen above, the OIS helps in determining the content and accordingly the precise
measure. For some reason there appear to be some errors connected to indicator 18, since the HRD
OP states a target of 2,000 while the SAR ambitiously raise the target to 20,000. The targets in the
MIS are in general misleading.
Furthermore, the SAR (as well as the MIS as it seems) has removed indicator 37 and 38. It has not
been possible to track the indicators even through the MIS. On the other hand, the OIS has identified
4 additional output indicators, of which one of them appears to be a revised indicator 38: “Number of
schools (secondary education institutions especially those for girls) equipped in the pilot provinces to
contribute to girls’ schooling and quality education”.
Unfortunately, the new indicator does not explicitly define the equipment needed to fulfil the objective;
are we talking about gender-specific toilet facilities, specific learning materials addressed to girls
(whatever that might be) or something else? What is quality education for girls? And how is quality
gender specific? The indicator is not precise enough which actually is quite surprising due to the solid
update of the other HRD OP named indicators (which appeared much undone in the HRD OP).
Otherwise, the Interim Evaluation have no further comments to the either the updated HRD OP-named
output indicators or the new indicators though we still wonder what happened to output indicator 19 on
action plans.
2.2 Measure 2.2
To improve the content and quality of Vocational and Technical Education
Specific Objectives:
•
To increase the quality of vocational and technical education and training addressing
curriculum, teachers, learning environment and school managers
•
To increase flexibility of the education system by facilitating vertical and horizontal
transitions between different types of programmes in the VET system
•
To make VET an attractive option for students
•
To promote cooperation between social partners, schools and VET centres and private
sector
164
Table 2.2.1 Output indicators on Measure 2.2
Count
43
44
45
46
Indicator
Number of
developed / revised
and implemented
modular programs in
secondary and post
secondary
vocational schools
by region, by training
institution / school
and by domain
Number of
comprehensive and
scheduled
awareness raising
events organised
with the cooperation
of private sector to
increase awareness
raising among the
sector by region and
by economic sector
Number of
partnership protocols
between vocational
institutions, social
partners and private
sector in 12 NUTS II
regions
Number of
secondary and postsecondary VET
schools with
improved capacity
and programs by
region and by
training institution /
school
Target
(2009)
250
vocational
branches
Target
(2011)
260
0 (except voluntary
campaigns conducted by
NGOs or private companies
nationwide).
Min. 30
42
Conference
reports
N.A (There is no such
collective protocols except
individual attempts which
might be determined when
the measure is
implemented).
30
32
MoLSS
Programme
Reports, Final
Beneficiary
Reports
N.A.
45
63
MoNE
Records of higher
VET schools of
the related
regional
universities ,
MoLSS
Programme
Reports, Final
Beneficiary
Reports
Baseline (2006)
42 ISCED-97
vocational/educational fields
and 192 vocational
branches developed under
SVET
Source of
Verification
Programmes
approved by
Board of
Education and
YOK, MoLSS
Programme
Reports, Final
Beneficiary
Reports
Source: Human Resource Development Operational Programme, second version
Table 2.2.2 Output indicators on Measure 2.1 as found in SAR
Count
43
44
Indicators
HRD OP
OIS
Number of
Number of modular
developed / revised
VET curricula
and implemented
developed and/or
modular programs
revised on Judicial
in secondary and
Practices,
post secondary
Agriculture, Maritime
vocational schools
and Health areas
by region, by
training institution /
school and by
domain
Number of
Number of
comprehensive and
communication
scheduled
strategies developed
awareness raising
through awareness
events organised
raising activities
with the cooperation
of private sector to
increase awareness
Baseline
42 ISCED-97
vocational /
educational fields
and 192
vocational
branches
developed under
SVET
0 (except
voluntary
campaigns
conducted by
NGOs or private
companies
nationwide).
Target
Expected
contribution
In the OP: 250
vocational
branches In the
OIS: 4 sets of
modular VET
curricula
Min. 30
165
Count
45
46
Indicators
raising among the
sector by region
and by economic
sector
Number of
Number of
partnership
partnership
protocols between
protocols between
vocational
vocational
institutions, social
institutions, social
partners and private partners and the
sector in 12 NUTS II private sector in 12
regions
NUTS II Regions
Number of
secondary and
post-secondary
VET schools with
improved capacity
and programs by
region and by
training institution /
school
47
48
49
50
Baseline
Target
N.A (There is no
such collective
protocols except
individual
attempts which
might be
determined when
the measure is
implemented).
N.A.
In the OP:30 In
the OIS: Minimum
20
Revised: Number of
teachers participated
in technical training
in the sector for
increased rate of
pedagogical and
professional
qualifications of
administrators and
teachers
N.A.
Revised: Number of
teachers participated
in trainings for
increasing the
quality of vocational
and career guidance
services in
secondary
vocational education
institutions and postsecondary higher
schools improved
N.A.
In the OIS: 1800
teachers (350
teachers from
Post Secondary
Higher VET
Schools and 1450
teachers from the
MoNE)
participated in the
technical training
in the sector
In the OIS: 700
secondary school
and 200 postsecondary higher
school teachers
Revised: Number of
modular VET
curricula developed
and/or revised on
Judicial Practices,
Agriculture, Maritime
and Health areas
Revised: Number
of teachers
participated in
Teacher Training in
line with the newly
updated curriculum
Revised: Number of
QASs in line with the
European Quality
Assurance
Reference
Framework for VET
(EQARF) developed
and piloted In the
selected 20 postsecondary higher
schools and 30
vocational
educational
institutions and
revised according to
Expected
contribution
In the OP: 45 In
the OIS: 4 sets of
modular curricula
N.A.
1500 Teachers
N.A.
In the OIS: 30
vocational high
schools) (20 postsecondary higher
vocational
schools)
166
Count
51
Indicators
the results of the
piloting.
Revised: Number of
departments
provided with
equipment support
that are providing
training in the fields
of Judicial Practices,
Agriculture, Maritime
and Health in the
selected 20 pilot
post-secondary
higher vocational
schools and
METARGEM
Baseline
N.A.
Target
Expected
contribution
In the OIS: 30
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
2.2.1 Interim Evaluation’ comments to Output Indicators underneath Measure 2.2
The Interim Evaluation recognizes again the revision done by the OIS, which all, more or less, appear
to be highly justified and needed. One single comment should be that the revised indicator 22,
Number of communication strategies developed through awareness raising activities, still appear
slightly weak and could use some more sharp and precise definitions: Perhaps there is a link
somewhere between awareness raising activities and development of communication strategies (if not
directly being the same); in that case it would certainly help to know.
Otherwise, there appear no practical objections to the revised indicators except for the number of
indicators which again appear to be way too many. The TAT had no access to data, but based upon
experience with indicators there should not be any practical hindrances for obtaining the data in case
the information lines can be established.
It might be worth to note that some of the newly revised indicators do not apply to the HRD OP
Regions alone indicating that in case regional data should be provided and relevant for the indicators,
the stakeholder should find ways to ensure data from all NUTS 2 regions of Turkey.
II.3 Priority axis 3
To increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting
lifelong learning and encouraging investment in human resources by enterprises and workers
3.1 Measure 3.1
Promoting the development and implementation of coherent and comprehensive strategies for
lifelong learning
Specific Objectives:
•
To provide basic skills and competences especially for the disadvantaged and women
•
To improve quality of non-formal training in line with the LLL perspective
•
To develop a comprehensive new approach to valuing learning to build bridges between
different learning context and learning forms and to facilitate access to individual
pathways of learning.
•
To strengthen the existing VET institutions for organizing VET courses in line with LLL
strategies and according to the ECVET principles.
167
•
•
•
To support establishment of Vocational Standards Development, Testing and Certification
Centres (VOC-TEST) as accredited bodies of vocational standard development, testing
and certification under the monitoring and supervision of Vocational Qualifications
Authority.
To support the establishment of National Qualification System
To promote the linkage between LLL, education system and labour market
Table 3.1.1 Output indicators on Measure 3.1
Count
Target
(2011)
8000
Source of
Verification
Enrolment statistics
Attendance rates,
MoLSS Programme
Reports, Final
Beneficiary Reports
Min. 11
80
15000
18000
MoNE records, VQA
records, vocational
standards issued,
MoLSS programme
reports, final
beneficiary reports
Certificates issued,
MoLSS Programme
Reports, Final
Beneficiary Reports
Indicator
Baseline (2006)
Target (2009)
Participants certified and
tested in terms of
vocational knowledge
and skills in 10 VOCTEST by region and by
VOC-TEST Centre
Number of Vocational
Standards developed by
VOC-TEST Centres and
endorsed by VQA by
VOC-TEST Centre
0
5000
Participants
(50 % Female,
50 % male)
0
54
Number of participants
certified on basic skills
courses by region and
by VOC-TEST Centre
84662
55
Number of people who
participated in
employability courses
provided by networks
NA (There was
no such a
network before)
M: 900
F: 900
56
Number of teachers who
participated in the
technical training in the
sector by region
NA (Data will be
collected when
the measure
implemented)
900
57
Number of people who
participated to
internships provided by
these networks (60-65%
of participants are
female, 40-35% are
male.20-25 % of
participants are
graduates of primary
education, 80-75% are
secondary education)
NA (There was
no such a
network before)
M: 600
F: 600
M: 1000
F: 1000
MoNE Records,
Work Place Records,
MoLSS Programme
Reports, Final
Beneficiary Reports
58
Number of teachers
trained on new methods
of monitoring, evaluation
and measurement for
LLL
NA (Innovative
approach for
Turkey, there is
no meaningful
data)
900
1500
59
Number of additional
training activities on
basic skills and further
development of basic
skills for the adults,
particularly for women
Attendance sheets
Surveys, MoNE
Reports, MoLSS
Programme Reports,
Final Beneficiary
Reports
Attendance sheets
Surveys, MoLSS
Programme Reports,
Final Beneficiary
Reports
52
53
5127
860 +
M: 1500
F: 1500
1500
1435
Participant List,
MoLSS Programme
Reports, Final
Beneficiary Reports
Participant List
168
Count
60
Indicator
Baseline (2006)
Target (2009)
Number of networks
among formal and nonformal
educationproviders,
business life, NGOs and
social partners providing
courses for increasing
employability of people
0
60
Target
(2011)
Source of
Verification
Protocols signed
Courses opened
Statistics on
employed trainees
(comparisons
between before and
after the courses),
MoLSS Programme
Reports, Final
Beneficiary Reports
90
Source: Human Resource Development Operational Programme, second version
Table 3.1.2 Output indicators on Measure 3.1 as found in SAR
Count
52
53
Indicators
HRD OP
OIS
Participants certified
3000 participants
and tested in terms of
certified and tested
vocational knowledge
in terms of
and skills in 10 VOCvocational
TEST by region and
knowledge and
by VOC-TEST Centre skills in 10 VOC
Test Centres.
Number of Vocational
Minimum 11
Standards developed
Occupational
by VOC-TEST
Standards
Centres and endorsed endorsed by VQA
by VQA by VOCTEST Centre
54
Number of
participants certified
on basic skills
courses by region and
by VOC-TEST Centre
55
Number of people
who participated in
employability courses
provided by networks
Number of teachers
who participated in
the technical training
in the sector by region
Number of people
who participated to
internships provided
by these networks
(60-65% of
participants are
female, 40-35% are
male.20-25 % of
participants are
graduates of primary
education, 80-75%
are secondary
education)
Number of teachers
trained on new
methods of
56
57
58
Baseline
Target
0
5000 (3000 in
OIS) participants
(50% female,
50% male)
0
Min. 11
Trainings on basic
skills and further
development of
basic skills for the
adults particularly
for women in formal
and non-formal
vocational
institutions in 12
NUTS II Regions
provided.
1800 participants in
employability
courses by these
networks
900 teachers
participated in the
technical training in
the sector
1200 participants to
internships
provided by these
networks
84662
NA (data will be
collected when
the measure
implemented)
NA (There was
no such a
network before)
900
Revised: Number of
teachers trained on
new methods for
NA (Innovative
approach for
Turkey, there is
900
NA (There was
no such a
network before)
15000
Expected
contribution
285
15
962+120: 1082
App. (-) 13000
M: 900, F: 900
306 App. (-)
1500
118
M: 600 F: 600
131
34
169
Count
59
60
Indicators
monitoring, evaluation monitoring,
and measurement for
evaluation and
LLL
measuring LLL
activities that are
developed, tested
and implemented.
Revised: Number
Number of additional
training activities on
of “trainings on
basic skills and further basic skills and
development of basic
further development
skills for the adults,
of basic skills for
particularly for women the adults
particularly for
women in formal
and non-formal
vocational
institutions in 12
NUTS II Regions
provided.”
Number of networks
Number of focus
among formal and
groups networks
non-formal
established
educationproviders,
business life, NGOs
and social partners
providing courses for
increasing
employability of
people
Baseline
no meaningful
data)
Target
5127
0
Expected
contribution
860
60
17
26
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational
Programme, 2010, Operating Structure, MoLSS, dated June 2011
3.1.1 Interim Evaluation’ comments to Output Indicators underneath Measure 3.1
The OIS does not add new indicators. However, it redefines and clarifies the existing indicators. Not all
of the revisions have been equally clever; Indicators 55 to 57 in OIS are basically not indicators. They
are targets.
Otherwise, there appear to be no formal objections to the indicators on behalf of the Interim
Evaluation.
3.2 Measure 3.2
To increase adaptability of employees and employers by investing more in human capital
Specific Objectives:
•
To improve adaptability of employees in SMEs by facilitating their access to training, in
particular for low-skilled workers,
•
To improve adaptability of employers in SMEs by designing and disseminating innovative
and sustainable forms of work organization, which support labour productivity and quality
at work,
•
To increase training capacities of social partners, NGOs, VET Institutions and public
bodies in order to establish capacity to facilitate adaptability of the employees and
employers
170
Table 3.2.1 Output indicators on Measure 3.2
Count
61
62
63
64
Indicator
Baseline (2006)
Number of employees
who participated in
trainings on work
organisation following
training needs analysis
Number of employers
who participated in
trainings organized
following a training
needs analysis
N.A.
N.A.
Number of social partner
staff participated in
training for trainers
Number of institutions
(NGOs, trade unions,
public institutions etc.)
benefited from
consultancy services on
the improvement of the
adaptability of employers
and employees for
social partners,
universities, public and
private VET Institutions,
NGOs, public bodies
and enterprises
N.A.
N.A.
Target (2009)
1200
1200
600
50
Target
(2011)
Source of
Verification
3400
Records of Final
Beneficiaries
MoLSS Programme
Reports
2400
Records of Final
Beneficiaries
MoLSS Programme
Reports
1800
Records of Final
Beneficiaries
MoLSS Programme
Reports
130
Records of Final
Beneficiaries
MoLSS Programme
Reports
Source: Human Resource Development Operational Programme, second version
Table 3.2.2 Output indicators on Measure 3.2 as found in SAR
Count
61
65
66
Indicators
HRD OP
OIS
Number of employees Revised: Number
who participated in
of employees
trainings on work
participated in
organisation following
trainings
training needs
analysis.
Revised: Number
of study visits by
SME employees
who have who have
participated and
successfully
completed the
trainings to SMEs in
an EU country
Revised: Number
of SMEs visited in
developed and
industrialized
provinces with the
participation of
employers who
have participated
and successfully
completed the
trainings
Baseline
Target
NA
Expected
contribution
OP: 1200 OIS:
3000 (total
number of
employees preferably of
SMEs whose
employers also
participate in
trainings)
OIS: max. 120
man/day in total
at least 25 SMEs
171
Count
67
68
69
62
70
71
72
73
74
Indicators
Revised: Number
of staff of
chambers, social
partners, Provincial
Employment and
Vocational Training
Board staff and
ÇASGEM staff
increased their
training capacities
to be trainers
Revised: Number
of gap analysis and
trainings needs
analysis at
company level in
each 15 growth
centres
Revised: Number
of vocational school
teachers
participated in onthe- job trainings
Revised: Number
Number of employers
who participated in
of employers
trainings organized
participated in the
following a training
trainings
needs analysis
Revised: Number
of study visits to
SMEs in EU with
the participation of
employers
Revised: Number
of workshops by the
participation of
employers from
developed regions
at sectoral level.
Revised: Number
of visits to selected
SMEs in developed
and industrialized
provinces with the
participation of
employers and
employees
Revised: Number
of models of
innovative and
sustainable forms
of work
organisation for
SMEs which have
participated in and
successfully
completed the
trainings
Revised: Number
of SME provided by
training and for the
establishment of
support and
coordination
centres within the
chambers and 1
Baseline
Target
OIS: 600
NA
NA
Expected
contribution
at company
level: OIS: 15
also, in at least
15 most
developed
provinces of
Turkey
200
OP: 1200
OIS: 2000
NA
2 study visits,
max. 80 man/
day in total
NA
5
NA
at least 25 SMEs
NA
at least 5 models
for 20 SMEs
NA
OIS: 600
172
Count
75
63
76
77
64
Indicators
within TOBB in
each of which nonkey experts and
chamber staff will
work
Revised: Number
of awareness
raising meetings for
raising the
awareness of
employers and
employees on the
need for more
training and
consultancy for
increasing
adaptability and
competitiveness
Number of social
Revised: Number of
partner staff
chamber staff
participated in training accredited as
for trainers
trainers
Revised: Number
of study visits by
chamber staff and
centre coordinators
to the training and
consultancy centres
established by a
chamber in the EU
Revised: Number
of online database
including
information on
stakeholders,
resources, the
training needs,
training providers,
announcements
regarding career
opportunities,
decisions of
Provincial
Employment and
Vocational
Education Boards
that will serve for
chambers, SMEs,
civil society
organisations and
vocational schools
is established
Number of institutions
(NGOs, trade unions,
public institutions etc.)
benefited from
consultancy services
on the improvement
of the adaptability of
employers and
employees for social
partners, universities,
public and private
VET Institutions,
NGOs, public bodies
and enterprises
Baseline
Target
Expected
contribution
NA
15
(approximately
1500 employers
and 1500
employees will
participate in
these meetings)
NA
OP: 600 OIS: at
least 6 chamber
staff accredited
as trainers
2 study visits
(max. 60
man/day in total)
NA
NA
NA
1
50
173
Count
78
Indicators
Revised: Number
of web- sites
established for
enhancing the
cooperation and
coordination among
relevant
stakeholders
Revised: Number
of conferences
established for
increasing industryuniversity
cooperation
79
Baseline
Target
NA
one
NA
one
Expected
contribution
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational
Programme, 2010, Operating Structure, MoLSS, dated June 2011
3.2.1 Interim Evaluation’ comments to Output Indicators underneath Measure 3.2
The OIS add or revise the indicators of the HRD OP adding some 15 new indicators into the
framework41.
The Interim Evaluation has found no reason to make explicit comments to any of the revised indicators
in the OIS.
II.4 Priority axis 4
To promote an inclusive labour market with opportunities for disadvantaged people, with a
view to their sustainable integration into the labour force and combat all forms of
discrimination in the labour market
4.1 Measure 4.1
To increase employability of disadvantaged persons, facilitate their access to labour market,
and eliminate barriers for their entrance into labour market
Specific Objectives:
•
To increase employment and labour force participation rate of disadvantaged persons
•
To facilitate disadvantaged persons’ access to labour market
•
To eliminate barriers for disadvantaged persons’ entrance into labour market
Table 4.1.1 Output indicators on Measure 4.1
Count
41
Indicator
80
Number of people
having participated in
guidance and
counselling services
81
Number of people
having benefited from
rehabilitation
programmes
Baseline (2006)
Target (2009)
Ex-prisoner and
ex-convicts = 188
for 3 years
(There is no
helpful data on the
people with
disabilities)
Ex-prisoner and
ex-convicts 114
people for 3 years
1712 people with
disabilities
Target (2011)
M: 2000
F: 2000
M: 3500
F: 3500
M: 1500
F: 1500
M: 3000
F: 3000
Source of
Verification
MoLSS Programme
Reports, Final
Beneficiary Reports
MoLSS Programme
Reports, Final
Beneficiary Reports
This time we have indicated the new added indicators by italics
174
Count
82
83
Indicator
Number of people
having participated in
courses in scope of
ALMPs
Number of people who
are reached through
the awareness raising
activities
Baseline (2006)
regarding SHÇEK
rehabilitation
centres for 3 years
in NUTS II
regions.
395 women with
disabilities, 1015
men with
disabilities, 12
women exprisoners, 93 men
ex-prisoners for 3
years under ALM
Project.
Target (2009)
1185 women
with
disabilities,
3045 men
with
disabilities,
36 women exprisoners, 279
men exprisoners
651 people are
reached in
Adıyaman,
Çankırı,
Gaziantep,
Kahramanmaraş
, Kastamonu,
Kayseri, Malatya,
Sinop, Şanlıurfa in
the scope of
Awareness
Raising activities
of Administration
for the Disabled in
2007
100000
Target (2011)
Source of
Verification
1985 women
with
disabilities,
MoLSS Programme
Reports, Final
Beneficiary Reports
5045 men
with
disabilities,
76 women
ex-prisoners,
479 men exprisoners
170000
Surveys, MoLSS
Programme
Reports, Final
Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
4.1.1 Interim Evaluation’ comments to Output Indicators underneath Measure 4.1
As noted in the theoretic introduction above output indicators are in general not that difficult to define
since they appear almost natural from the definition of the measures, actions and target groups.
Furthermore, output indicators are by nature almost always of quantitative nature, since the main
purpose of output indicators is to measure the direct effect of the effort; that is numbers.
Output indicators on Social Inclusion are no exception. The difficulties in determining Output Indicators
on Social Inclusion are connected with two things: 1) difficulties in defining the target groups and 2)
obtaining data.
In other countries, notably Member States of the EU, the discussion of the target groups continues and
basically never stops; it is a political topic, obviously. As examples Social Inclusion could easily be
fully-functional persons lacking basic reading abilities, it could be immigrants, even second generation
immigrants, and it could be unemployed in age groups of 55 and above. Some pessimistic researchers
(semi-opinion-makers) might argue that smokers tend to be Social Excluded these years, since some
enterprises follows the new-economics, puritan American business philosophy, has banned smoking
from the grounds, and where smokers cannot be deployed 42.
Accordingly, definitions differ across countries and perhaps quite rightly so. When it comes to the
definition of Social Excluded there is no golden rule, though a lot of international NGOs might think
differently and though UN have defined some minimum criteria. From a methodological, theoretic and
scientific point of view that discussion is often not an issue; the definition has become a matter of
taste.
42
And that could soon generate Social Exclusion by definition
175
The difficulties in Social Inclusion arise when it comes to obtain reliable data and statistics. We might
have a good idea of the target groups, the marginalised and social excluded, but often we do not keep
track of them – we might count them when the address public institution but otherwise we cannot see
where they live.
Turkey is obviously no exemption. Having defined the target groups to be mainly ex-prisoner, exconvicts, and people with disabilities, the next problem appears to find reliable data to constitute the
baseline where we would expect SSI to have some basic data 43.
The Interim Evaluation Report has no objections to the definition of target groups insofar a note within
the system explicitly defines what constitute disability44. We shall expect that there is a law framework
which clarifies the target group sharply. We shall assume that the prime Operation Beneficiaries
(MoLSS, İŞKUR, MoNE and SSI) can redefine the target groups in later programmes if needed.
Neither has the Interim Evaluation found reason to raise fingers of the data (targets and baselines).
The comment in the HRD OP on the new approach which is signalled in the baselines justifies a
baseline of “close to zero”.
The only comment which was found relevant is the strange gender difference in the target of Number
of people having participated in courses in scope of ALMPs 45; apparently, the measure favours men to
women.
4.2 Measure 4.2
Better functioning and coordination among the institutions and mechanisms in the field of
labour market and social protection particularly in order to facilitate the integration of
disadvantaged persons into the labour market
Specific Objectives:
•
To improve the institutional capacities of public service providers to disadvantaged
persons in their employment related services, and the functioning of existing mechanisms,
•
To ensure better coordination among public service providers to disadvantaged people,
including between central and regional/local authorities and in cooperation with NGOs, in
employment related services
Table 4.2.1 Output indicators on Measure 4.2
Target
(2009)
1500
Target
(2011)
N.A.
Source of
Verification
MoLSS Programme
Reports, Final
Beneficiary Reports
0
1
N.A.
2069 staff in SYDGM in
central and local level.
222 staff in SSI in
central level.
5500 in
total.
N.A.
MoLSS Programme
Reports, Final
Beneficiary Reports
MoLSS Programme
Reports, Final
Beneficiary Reports
Count
Indicator
Baseline (2006)
84
Number of staff of social
protection institutions and
NGOs who participated in
trainings on services related
to the disadvantaged
persons
Number of common
monitoring and analysis
system established
Number of staff of related
institutions and NGOs in that
area who participated in
trainings on the formation,
N.A. (There are no
meaningful data. Data
will be collected when
the measure is
implemented)
85
86
43
Unfortunately, the TAT received no data from SSI and background data from TUIK/TURKSTAT are simply not
detailed enough.
44 The definition of “disability” differs significantly across countries.
45 The target mentions 1185 women and 3045 men.
176
Count
87
Target
(2009)
Indicator
Baseline (2006)
usage and implementation of
central database.
1394 staff in İŞKUR.
20 staff in
Administration of
Disabled
683,269 disadvantaged
persons in SSI
database in NUTS II
level
161,097 disadvantaged
persons in SYDGM
project database.
20,626 people with
disabilities in the
database of
Administration for
Disabled in NUTS II
regions.
Number of disadvantaged
persons registered to central
database, monitoring and
analysis systems.
Target
(2011)
1,250,000
disadvanta
ged
persons
N.A.
Source of
Verification
MoLSS Programme
Reports, Final
Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
4.2.1 Interim Evaluation’ comments to Output Indicators underneath Measure 4.2
In general, the Interim Evaluation has no additional comments to the comments mentioned in subsection 9.4.1.1. The only very light comment could be that the target of 1,250,000 disadvantaged
persons under Output Indicator 87 raises an inferior question: What if you cannot find 1,250,000
disadvantaged persons in the named NUTS 2 Regions?
The question illustrates that somehow the indicator is strange since it pursue something which
basically is not the issue; the issue is to have the disabled persons registered (and not the number). It
might indicate that the target should be expressed as a fraction of total disabled persons.
The problem might be that we do not know the exact number of total disabled persons, but that fact
would just justify the irony of the question raised.
We will allow this comment to stay as a light recommendation; the recommendation is not vital.
II.5 Priority axis 5: Technical Assistance
5.1 Measure 5.1
Support for Programming, Management, Implementation, Monitoring, Control, Evaluation and
Dissemination Activities
Specific Objectives:
•
The main aim of this measure is to provide support to the OS in different steps of the IPA
process and to increase the capacity of the relevant central and local partners in the
related issues.
Table 5.1.1 Output indicators on Measure 5.1
Count
84
85
86
Indicator
Minimum number of trainings for
the staff of the Operating
Structure related to improvement
of knowledge and practice of IPA.
Minimum number of trainings on
the usage of MIS
Minimum number of trainings for
OS staff to improve their
Baseline
(2006)
N.A.
Target
(2009)
24
Target
(2011)
48
Source of
Verification
MoLSS Programme
reports
N.A.
9
19
N.A.
15
20
MoLSS Programme
Reports
MoLSS Programme
Reports
177
Count
87
88
89
Baseline
(2006)
Indicator
knowledge on the IPA-ESF and
differences between them
Minimum number of site visits for
the monitoring of the projects
Minimum Number of evaluation
activities
Minimum number of prepared
analyses, studies, strategy
papers, manuals etc.
Target
(2009)
Target
(2011)
N.A.
80
160
N.A.
2
3
N.A.
10
20
Source of
Verification
MoLSS Monitoring
and Information
System, project and
programme reports
MoLSS programme
reports
MoLSS Programme
Reports
Source: Human Resource Development Operational Programme, second version
5.1.1 Interim Evaluation’ comments to Output Indicators underneath Measure 5.1
The Interim Evaluation has no comments to the output indicators and their targets.
5.2 Measure 5.2
Support for development of absorption capacity of final beneficiaries
Specific Objectives:
•
The main aim of this measure is to improve absorption capacity of the final beneficiaries in
order to develop a project pipeline in the framework of HRD OP.
Table 5.2.1 Output indicators on Measure 5.2
Count
90
Indicator
Minimum number of
capacity building
activities (trainings,
workshops etc.)
Baseline (2006)
Available
Target (2009)
90
Target (2011)
140
Source of
Verification
MoLSS Programme
reports
Source: Human Resource Development Operational Programme, second version
5.2.1 Interim Evaluation’ comments to Output Indicators underneath Measure 5.2
The Interim Evaluation does not understand the indicator and even lesser the baseline.
Minimum number of capacity building activities does not make sense; it is simply not specific and
sharp and can subsequently hardly be measured. The indicators should specify the areas of expertise
which is sought improved for obvious reasons 46.
The baseline “available” is beyond understanding.
5.3 Measure 5.3
Information and publicity activities
Specific Objectives:
•
The objective of this measure is to assist the implementation of obligations arising from
Article 62 of IPA Implementing Regulation concerning the promotion of the HRDOP and
its operations and informing entities interested in receiving support from the HRDOP, as
well as the general public, about the opportunities provided by the assistance and its
outcomes.
46
From experience with previous output indicators it appears plausible to assume that the OIS (Operation
Identification Sheets) has specified and clarified which capacity building activities we are talking about. It has not
been possible to obtain access to the OIS within the time limit of this report.
178
Table 5.3.1 Output indicators on Measure 5.2
Count
91
Indicator
Minimum number of
information events
(seminars, workshops
etc.)
Baseline (2006)
Available
Target (2009)
50
Target (2011)
100
Source of
Verification
MoLSS Programme
reports
Source: Human Resource Development Operational Programme, second version
5.3.1 Interim Evaluation’ comments to Output Indicators underneath Measure 5.3
The Interim Evaluation refers to the comments in sub-section 5.2.1.
179
PART III: Result Indicators
Part III presents the introduction to the study of Result Indicators. However, some further notes to the
study appear necessary.
First, the induction to the study of indicators describes the important difference between Outcome and
Impact Indicators, which in the current HRD OP is merged to Result Indicators. Perhaps the merging
of the two can be justified since it appears easier to explain, the Interim Evaluation finds reason to
believe that the merging has generated some methodological shortcomings to future evaluations by
the merge. Accordingly, the Interim Evaluation will try to categorize the named indicators into Outcome
and Impact indicators.
Second, while the study of the relevance of output indicators can be done fairly easy, simply because
output indicators are so closely linked to final beneficiaries, the study of the relevance of Outcome and
not least Impact Indicators becomes far more exquisite: With respect to Outcome Indicators we have
to incorporate the complete potential participants of the target groups, and with respect to Impact we
potentially have to look at the overall welfare for the regions in question.
With respect to the study of the relevance of the impact indicators, the main point is to find the link to
the priorities and its measures, of course, but basically even that might not be enough. In most cases it
would be far more obvious to understand the more global objective of Human Resources Development
in general and notably for the country and the regions in question. We shall return to these last
aspects in the final sections of this Part which summarizes the findings of the study of the Indicators of
the HRD OP.
In the evaluation of Result Indicators we basically do not need data to study the relevance. Referring
to the general introduction to the Interim Evaluation Report and the methodology notes in the
introduction to the study on Indicators in this Annex, we regret that we have no data to study the
quality of the data, and in order to create time series and cross-section analysis to study the efficiency
of the measure, the indicators, and their targets, which not least would have been important for Impact
assessment. However, when it comes to Outcome and more so Impact, the delayed progress of the
implementation of the HRD OP would in any case not have provided us means to make impact
assessments.
The study of the single result indicators follows a set of “tests” based upon the following questions:
1. Is the indicator understandable and measurable from common practice?
2. Is the indicator an Outcome or an Impact Indicator?
3. Does the Indicator mirror or reflect a relevant and sustainable process?
4. Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and the
social and economic conditions facing the NUTS 2 Regions of HRD OP?
5. Are data available and are the data consolidated ensuring means to do solid evaluation?
The study of each indicator is rounded by:
6.
Overall conclusion and preliminary recommendation
Not all indicators stand the test; in that case the study of the indicator will be terminated by a
conclusion / recommendation.
180
III.1 Priority axis 1
To attract and retain more people in employment, particularly by increasing labour force
participation of women, and decrease unemployment rates especially for young people
III.1.1 Measure 1.1
To promote women’s participation into the labour market, and increase female employment,
including those formerly employed in agriculture
Specific Objectives:
•
To improve employability of women
•
To support women entrepreneurship
•
To diminish cultural and other obstacles that hinder women’s participation in labour
market
Table 1.1.1 Result indicators on Measure 1.1, Priority axis 1
Count
1
2
3
Target
(2009)
Target
(2011)
95 %
95%
İŞKUR Job Seeking
Records, MoLSS
Programme Reports,
Final Beneficiary Reports
27 % (According
35 %
to İŞKUR data for
3 years in NUTS II
regions)
23%
25.6 % (According 30 %
to KOSGEB and
İŞKUR data)
20%
İŞKUR Job Placement
Records, SSI Records,
MoLSS Programme
Reports, Final Beneficiary
Reports
KOSGEB Records, SSI
Records, MoLSS
Programme Reports,
Final Beneficiary Reports
Indicator
Baseline (2006)
Increased rate of women having
entered into labour market
(having the qualification and
motivation to seek a job) after
participating in courses or
counselling
Increased rate of women
employed through the courses
and counselling
88 % (According
to İŞKURAIP
Project Data for 3
years)
Increased rate of women
entrepreneurs
Source of Verification
Source: Human Resource Development Operational Programme, second version
Table 1.1.2 Result indicators on Measure 1.1, Priority axis 1 as found in SAR
Count
1
2
3
Indicators
HRD OP
Increased rate of
women having entered
into labour market
(having the qualification
and motivation to seek a
job) after participating in
courses or counselling
Increased rate of
women employed
through the courses and
counselling
Increased rate of
women entrepreneurs
Baseline
Target
88 % (According
to İŞKURAIP
Project Data for 3
years)
95 %
Expected
contribution
2484
27 % (According
to İŞKUR data
for 3 years in
NUTS II regions)
25.6 %
(According to
KOSGEB and
İŞKUR data)
35 %
1094
OIS
30 %
928
PYE: 464
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational
Programme, 2010, Operating Structure, MoLSS, dated June 2011
181
Table 1.1.3 Output indicators on Measure 1.1, Priority axis 1 as found in MIS
Ref.
Indicator
1
2
3
Target
Increased rate of women having entered into labour market
(having the qualification and motivation to seek a job) after
participating in courses or counselling
Increased rate of women employed through the courses
and counselling
Increased rate of women entrepreneurs
Achieved
5,313
3,342
3,194
787
2,182
789
Source: Monitoring Information System, Operating Structure, MoLSS
1.1.1 Result Indicator no. 1
1. Is the indicator understandable and measurable from common practice?
The first step in the study is obviously to understand the indicator. The indicator reads: “Increased rate
of women having entered into labour market (having the qualification and motivation to seek a job)
after participating in courses or counselling”. In order to understand the complexity of the indicator it
appears necessary to decompose it.
First, we have to understand the term “entering the labour market” correctly. It must be a mistake, a
mistake which is repeated in the Measure; obviously, the authors must have meant “entering the
labour force”47. In other words the indicator reads, that we would be happy if women steps out of
inactivity and into employment or unemployment 48.
But the indicator does not stop there. It further requests a rate, or more sharply: “the rate of women
getting a job or registers as unemployed after participating in courses or counselling”. Again, we have
to make presumptions: What rate? What is the denominator? Is the rate the reciprocal of drop-out
rates (or failure rates) from courses or counselling49? Or does the rate measure the difference
between those women who get a job after the named support divided by women who obtain a job
without participating and without support? Or is it number of women getting a job or get unemployed
divided by number of inactive women? Presumably not the last if we should believe the targets, but
generally no answer is given; and the MIS only provides numbers50.
Next, and this point will be raised a couple of times in this chapter, the indicator mention a need for an
“increase”. Obviously, that must be a mistake. Indicators never speak of increase or decrease and
never give any signs of directions; that is the “job” of the baselines and the targets of course.
Finally, we have an even more gigantic problem in the formulation to the requirements to the women
who actually find a job (if that is what it means): Exactly how do we measure women who have the
qualification and motivation to seek a job? And yes, both are equally impossible, qualification and
motivation, though, admittedly, qualification perhaps can be measured to some extent, but appears to
be quite symbolic in this context where the women actually obtains a job.
Of course, we could alternatively assume that the authors mean “getting a job” but that would bring the indicator
to become the same as indicator 2.
48 It does not indicate that it has to be registered employment, which is a matter of Measure 1.3, but we basically
do not know, and perhaps that is intentionally.
49 E.g. if 6 women obtain a job after participating in a training course with 10 participants in total we have a
fraction of 0.6, indicating that we have a drop-out rate (or perhaps a failure rate) of 0.4. Does that mean that the
indicator, the rate, measure successes? Or should we go even further as presume that the word “rate” determines
“speed” or “growth”?
50 We have, as mentioned in Chapter 8 above, data on “number of persons having benefited from guidance and
counselling services by region (NUTS 2) and gender” from İŞKUR, but we still do not know what rate means.
47
182
Measuring “right” qualifications for the job is a dangerous task which only few labour market analysts
would embark upon. Just imagine asking your manager the following: “Tell me, Sir, do you think you
are qualified for your job, you are doing?” Of course, we could make screenings of all job places,
which could include collecting of CV’s and educational diplomas allowing us to do some assessment
of the qualification levels for certain job positions, but if this woman obtained a job from an employer
who willingly gave her the contract, would we say that she is not qualified?
In other words, using criteria like “qualification” and “motivation” in indicators should only be done with
reason and not least care. We have to know how to measure it, and in this case it is plain impossible.
Unless, of course, the participants end up being unemployed51.
Accordingly, the Interim Evaluation does not understand the indicator. Furthermore, it cannot be
measured since we do not know the dominator and since motivation cannot be tested. Even if ignored
these facts and tried to measure it, we would not be able to proof that the women actually have the
right qualifications for the jobs (still in case the participation is crowned with some success, that is). In
other words: The indicator is incomplete, unfinished, undone, and not thought trough.
2. Is the indicator an Outcome or an Impact Indicator?
Assuming that the right words should have been something in the neighbourhood of “number of
women getting a job or register as unemployed after participating in courses or counselling” the
indicator can qualify to be an Outcome Indicator in two specific cases:
1. Insofar the purpose of the measure is to increase the number of registered women, or
2. Insofar we talk of courses or counselling in a very broad sense. The broad sense indicates a
necessity that the measures relates to courses or counselling outside the scope of HRD OP,
though still within the NUTS 2 Regions.
The first cause is basically the purpose of Measure 1.3 as mentioned, and it does not include women
who find a job in the informal sector. But then again, support of the informal sector has never been a
real target for any public measures. So, perhaps the indicator tries to seduce women to become
registered basically and perhaps as a sort of step 1 leaving the goodies to Result Indicator 2 for those
who obtain a job. If so, it would be far more feasible to name the indicator by its right name: Number of
formerly inactive women registered as unemployed”. But that is basically not what the indicator say,
and still it simply does not make sense to talk of courses and counselling then.
In case the indicator aims at courses or counselling in a very broad sense the measure more clings to
training of staff of İŞKUR as well as to ensure better trainers (or so), hoping to increase the quality of
the services provided and thereby ensuring a higher number of women who actually obtains a job.
Accordingly, we have to look at the measure, which reads: “To improve employability of women”. But
that is not enough, since we further have to understand the basic actions (the projects and their
contents) in order to see if they are designated to improve the quality of the courses or counselling.
Exactly that has not been possible within the short time of this evaluation project, but having a quick
glimpse at the output indicators under Measure 1.1 it appears quite obvious that we are counting
participants – and not counting staff and teachers trained.
That leaves only one conclusion: The so-called Result Indicator, even if we assume that we
understand it, is neither an Outcome nor an Impact Indicator. Changing the indicator into a fraction or
a rate does change nothing mainly since the indicator so clearly tries to ensure that the training
courses or the counselling service is improved for the selected few who obtain the benefit thereof.
51
Which hardly can be a priority, or what?
183
It is obviously just another Output Indicator if anything 52, though presumably an Output Indicator in a
slightly wider scope since it might incorporate other actions taken by İŞKUR. It simply measures the
number of women who circles through the public funded Active Labour Market Measures and get
through it with success53.
In order to qualify as an Outcome indicator we would have required the indicator to incorporate the
measure which reads: “To promote women’s participation into the labour market, and increase female
employment, including those formerly employed in agriculture” as well as the Priority “To attract and
retain more people in employment, particularly by increasing labour force participation of women, and
decrease unemployment rates especially for young people” 54. In that case, we might have found it
more indicative to look for labour force participation rates or perhaps even employment rates by
gender, where the Outcome Indicator could specify “women”.
Preliminary recommendation55: Redefine the indicator or perhaps delete it.
An obvious doable alternative would be a workable and highly practical well-defined
Outcome Indicator of “Labour Force Participation Rate for Women” or perhaps
“Employment rate for women”
In other words, assuming that the Interim Evaluation are right in its search for Outcome and Impact
Indicators amongst the Result Indicators of the HRD OP, the study of the first indicators was fairly
disappointing. On the other hand, the Interim Evaluation believes that the above comments expressed
through a light tone of almost “oral writings” ensure better understanding of how to define indicators in
the future. The Interim Evaluation will, as will be seen from the following sections below, bring the
issue up front in the final recommendations in the end of this very chapter.
1.1.2 Result Indicator no. 2
1. Is the indicator understandable and measurable from common practice?
“Increased rate of women employed through the courses and counselling” is definitely more sharply
formulated that indicator 1 above. However, the points about the words “rate” and “Increased” still
remains; we simply do not know what the denominator is, but we would presume it to be the total
number of participants in “courses and counselling” 56.
2. Is the indicator an Outcome or an Impact Indicator?
Having redefined the indicator to measure “Number of women employed after participating in courses
or after having received counsel divided by total number of women participating in courses and
counselling” we stand in the same dilemma as under indicator 1 above: Are we into measuring the
quality of the training courses / the counselling service, or do we have a specific target group to reach?
The “if anything” is not just another comment to the difficulties to see what the HRD OP stipulates to measure,
but also the fact that measuring “number of women getting a job after participating in courses or counselling” quite
easily could be valued to be slightly out of the scope of the actions taken within the projects.
53 Though still with a focus on improving the “systems” / institutions rather than the situation for the target group
as mentioned below.
54 The priority clearly states “Labour Force participation” which brings the discussion of “entering the labour
market” into perspective. And, right, now, after the study of the first indicator, we are in doubt of the meaning of
Measure 1.1; it would have been reasonable to suggest that at least part of the objective / priority / goal of the
HRD OP was to help decreasing the number of inactive, particularly aimed at women (noting the social structure
within the HRD OP Regions).
55 Throughout this section we use preliminary recommendations only since additional studies of the indicators
from a more overall viewpoint follows below. The Chapter will conclude by stating final recommendations
56 Well, we did not dare to mention it under indicator 1, but basically in that case it would be wrong to use the term
“rate”; “Share” would be a more proper term.
52
184
Well, women in general cannot be the target group; that would be far too global, of course, but what
about the dominator of total women having benefited from the public service? That would be logic if we
keep our eyes focused on the indicator alone. But recall that the measure talks of “promoting women's
participation into the labour market (labour force?), and increase female employment, including those
formerly employed in agriculture”. The measure speaks for a target group of women OUTSIDE the
labour force, obviously. So, the question would be: Would women outside the labour force benefit from
women obtaining a job after being trained? Obviously not; there is simply not a single element of
sustainability in-built in the indicator.
And that leads us back to the same story and the same conclusion as under indicator 1: Either the
indicator speaks of the quality of the service or it is not an Outcome indicator.
The recommendation under indicator 1 remains.
1.1.3 Result Indicator no. 3
1. Is the indicator understandable and measurable from common practice?
“Increased rate of women entrepreneurs” is plain talk in case we ignore the word “increase” and
assume that we know what an entrepreneur is. In common language for daily use we believe that
entrepreneurs own and run their own businesses. We shall accordingly presume that women
entrepreneurs are women who start their own business as owners 57. Self-employed, that is58.
The rate is not specified and the denominator could (again) be a number of things: Women in general /
female working age population (15-64) / women in labour force / total employed women / male
“entrepreneurs” etc.59. We shall presume that we speak of total employed women 60.
2. Is the indicator an Outcome or an Impact Indicator?
Basically, the entrepreneurs are not specifically mentioned as prime target in the priority and the
measure, so it tends to be difficult to assess the logic and the relevance of the indicator. Though it
appears to be a long shot to recruit new entrepreneurs among a group of women outside the labour
force it appears to make sense to encourage entrepreneurship among women. In that case it would
correct to place it as an Outcome indicator.
3. Does the Indicator mirror or reflect a relevant and sustainable process?
Ensuring encouragement for starting own business, including ensuring that the basic platform for
doing that is in place (in terms of financial support, legislative advisory, insurance systems, etc.), is
always relevant, perhaps even sustainable though that might take some time to show. But of course,
that depends on content which requires a far more qualitative indicator.
57
Does that exclude women elected to be chairman of the board of shareholders and women who are deployed
as managing or executive director? Presumably the latter but not the former if we should understand the term
“entrepreneurs” correctly.
58 Which normally includes freelancers (e.g. journalist, photographers, consultants, models, actors, etc.)
59 Likewise, the indicator do not clarify “full-time” versus “part-time” which easily could be essential in start-up of
enterprises.
60 Since that would be the most logic in accordance to international practice. However, that does not seem
plausible from the baseline and the target; can it really be true that 25 per cent of total women in employment are
self-employed?
185
If the main new enterprise start-ups emerge within a production or a market with no or limited demand,
the initial benefits might soon turn into a disaster, not just on a private account but also for the local
economy. Examples could be small-time shops and small-time craftsmanship (e.g. home-based
knitting ware and small-time tourist merchandise). If the effort, however, are directed into sub-sectors
with larger potentials and excess demand, the effort is alternate prosperous.
To answer that question we need far more qualitative information than given by the indicator, which
seen from an evaluation point of view it is basically not a requirement. The only answer to the question
of sustainability is “time must show”, while the answer to the question of relevance appears to be
positive, since stimulating enterprise start-ups is generally reckoned to beneficial for the society
(international experience)61.
4. Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and the
social and economic conditions facing the NUTS 2 Regions of HRD OP?
The relevance of the indicator subject to the Priority, the measure, the specific objectives, and the
social and economic conditions facing the NUTS 2 Regions of HRD OP, is however far more
questionable. The following table summarizes the framework of the indicator (Priority, Measures, and
Objectives) and tries to provide some brief comments to each of the elements within.
Framework
1) Attract and retain more people in
employment
2) Increase labour force participation
of women
3) Decrease unemployment rates
especially for young people
4) Promote women’s participation into
the labour market (which we
understand as Labour Force)
5) Increase female employment
6) Include those formerly employed in
agriculture
7) Improve employability of women
8) Support women entrepreneurship
9) Diminish cultural and other
obstacles that hinder women’s
participation in labour market
Relevance of the Indicator
Assuming that the recruitment of women comes from
inactivity we can speak of “attracting”. It is not evident
that entrepreneurship provide means of sustainability
for those who succeed, though
Assuming that the recruitment of women comes from
inactivity: Yes
If not: No.
Assuming that the recruitment of women comes from
unemployment: Yes. No long term effect is
guaranteed, though
If not: No effect and not relevant
Assuming that the recruitment of women comes from
inactivity: Yes
If not: No
Assuming that the recruitment of women comes from
inactivity: Yes
It depends. The indicator do not exclusively focus on
that segment
Assuming that the recruitment of women comes from
inactivity: Yes
If not: No. The indicator is not directed towards
measure of employability
Yes, that is the idea
It sounds plausible on labour markets mainly
dominated by men
Check
?
()
()
()
()
?
()


61
Again, and as mentioned, that could be questioned (and perhaps it should) in case the new enterprises do not
stand a fair chance on a private market. Somehow and somehow strangely, the question of demand is hardly ever
addressed.
However, approximately 10 years ago EC Commission launched a new indicator on “Survival rates of newly born
enterprises” which is defined as “Newly born enterprises of year n that are still active in year n+3”. The indicator
was former part of European Employment Strategy guideline.
We will not recommend using this indicator here; basically enterprise-start-ups are not the prime aim of the HRD
OP and its Priorities.
186
Accordingly, the main question is the extent the new entrepreneurs are recruited from inactivity or at
least either the pool of unemployed women. But even when the recruitment comes from the pool of
employed the indicator appears to be accepted from a very general point of view.
The reservation from the Interim Evaluation is solely of more qualitative, and perhaps more subjective
nature; 1) in case the new entrepreneurs are former employed a large part of the overall intention of
the Priority appear to be lost, and 2) in case some kind of sustainability is not directly ensured from
other measures and actions (do the new enterprises survive?), the net impact might soon turn to
become negative.
5. Are data available and are the data consolidated ensuring means to do solid evaluation?
The fifth and final test is data availability. Unfortunately data is not available according to İŞKUR62. The
same conclusion has been given by the Operating Structure in MoLSS.
However, it should be possible to develop some basic raw indicators from the Household Labour
Force Survey conducted by TÜİK/TURKSTAT. A set of data is available from the internet and called
“Employment by status in employment [15 years old and over-thousand people]: Employer and own
account worker” which series provides data for all NUTS 2 regions of Turkey since 2004 63.
However, the Interim Evaluation is not sure that the data is solid enough which concern includes a
denominator, and furthermore it was not possible to provide data on gender from TUIKs home pages.
Accordingly, we recommend the Operating Structure to contact TÜİK/TURKSTAT in order to find
means to obtain meaningful data series on the specific indicator.
Wherever data from KOSGEB and İŞKUR is available, these data sets, which will be un-official, can
be used for quantitative validation purposes 64.
By means of LFS the evaluators can further obtain data from other NUTS 2 regions providing a solid
fundament for measuring the effect.
6. Overall conclusion and preliminary recommendation
The Interim Evaluation has no objection towards the Outcome Indicator, except from some minor
reservations. The reservations are: 1) in case the new entrepreneurs are formerly employed a large
part of the overall intention of the Priority appears to be lost, and 2) in case some kind of sustainability
is not directly ensured from other measures and actions (do the new enterprises survive?), the net
impact might soon turn to become negative.
The Interim Evaluation recommends the Operating Structure to contact TÜİK/TURKSTAT to find ways
to obtain meaningful and consolidated data based on LFS.
See table 1 in this Annex: The response from İŞKUR.
LFS data series
64 The note here concerns a methodological note on the substance of data generated from administrative records;
as a general rule of thumb the more we move towards Outcome and not least Impact assessment the
requirement to the statistics becomes of paramount importance: Never use anything else than official statistics for
measures of Impact. With respect to Outcome we can use specially designated surveys and perhaps
administrative records, where the administrative records are able to feed in on official statistics (qualitative) or
otherwise be used as control. Recall, that administrative records are NOT validated (in terms of terminology we
can hardly call administrative record “statistics”, though that word is being used widely in some central
administrations).
62
63
187
1.1.4 Feedback from survey on indicators among the stakeholders on Result Indicator 1-3
What is the problem of the indicators under this measure, if any?
Share
12.7%
Net impact cannot be measured since Impact can arise from other factors in the
environment
6.3%
Data is not available
7.9%
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by
submission of reports, etc.)
14.3%
Data differs from region to region (not standardized)
9.5%
Data are not systematically collected (e.g. timing differs between projects and/or
regions)
9.5%
Data is only generated from on-going projects (we cannot see what status are in
locations without projects, before the project is launched, and after the project
has been concluded)
7.9%
The indicator does not mirror the real target / is not relevant
4.8%
The target group is not clearly defined
11.1%
There is no baseline data (we cannot see and measure improvements)
9.5%
The indicator is too complicated to calculate (if based upon aggregated data)
6.3%
Other (please specify)
N
8
4
5
9
6
6
5
3
7
6
4
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above
indicate the stakeholders view on where the main problems lie65.
III.1.2 Measure 1.2
To increase employment of young people
Specific Objective:
•
To promote integration of young people into the labour market
Table 1.2.1 Result indicators on Measure 1.2, Priority axis 1
Count
4
5
Target
(2009)
Indicator
Baseline (2006)
Increased rate of young people
employed following the courses,
internship and counselling
activities
In average 22.3 %
according to the
İŞKUR data in
NUTS II regions.
30 %
Increased rate of young
entrepreneurs
7.5 % According
to KOSGEB data
in NUTS II regions
15 %
Target
(2011)
Source of Verification
23% İŞKUR Job Placement
Records, SSI Records,
MoLSS Programme
Reports, Final Beneficiary
Reports
15% MoIT Records, SSI
Records, MoLSS
Programme Reports,
Final Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
Table 1.2.2 Result indicators on Measure 1.2, Priority axis 1 as found in SAR
Count
4
5
Indicators
HRD OP
Increased rate of young
people employed
following the courses,
internship and
counselling activities
Increased rate of young
entrepreneurs
Baseline
Target
30 %
Expected
contribution
4152
PWE: 35%
PWE: 1094
15%
464
OIS
In average 22.3
% according to
the İŞKUR data
in NUTS II
regions.
7.5 % According
to KOSGEB data
in NUTS II
regions
PWE: 30%
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational
Programme, 2010, Operating Structure, MoLSS, dated June 2011
65
We agree that the questions should be raises to each and every indicator, but that would have created a far too
long questionnaire.
188
1.2.1 Result Indicator no. 4
The Result indicator is basically the same as Result Indicator no. 2 above. Accordingly, we refer to the
comments noted in subsection 10.1.1.2. The preliminary recommendation from subsection 10.1.1.2
remains.
1.2.2 Result Indicator no. 5
The Result indicator is basically the same as Result Indicator no. 3 above. Accordingly, we refer to the
comments noted in subsection 10.1.1.3. The preliminary recommendation from subsection 10.1.1.3
remains.
With respect to data we recommend to contact TÜİK/TURKSTAT in order to obtain data by both
gender and age.
1.2.3 Feedback from survey on indicators among the stakeholders on Result Indicator 45
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by
submission of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or
regions)
Data is only generated from on-going projects (we cannot see what status are in
locations without projects, before the project is launched, and after the project
has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Share
N
12.1%
4
12.1%
4
15.2%
5
9.1%
3
12.1%
4
12.1%
4
9.1%
0.0%
6.1%
6.1%
3
0
2
2
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.1.3 Measure 1.3
To promote registered employment
Specific Objective:
•
This measure aims to promote registered employment through strengthening the capacity
of social security institution, provision of awareness raising and ensuring coordination
among relevant public institutions and social partners.
Table 1.3.1 Result indicators on Measure 1.3, Priority axis 1
Count
6
Target
(2011)
Indicator
Baseline (2006)
Target
(2009)
Decrease in the rate of
complaints (advice slip) on the
unregistered employment to the
Insurance Inspection Board
2432 individual
advice slips
- 15 % = -%5 =
2067
2310
Source of Verification
SSI Insurance Inspection
Board Records, MoLSS
Programme Reports,
Final Beneficiary Reports
189
7
8
Increase in the number of
bilateral projects on promoting
registered employment between
the relevant partners
Increase in the rate of
registered people in 43 city
centres through the bilateral
projects and local partnerships
N.A. (Data will be 12
collected when the
measure is
implemented)
66.3 %
66.6 %
20
MoLSS Programme
Reports, Final Beneficiary
Reports
66.6%
MoLSS Programme
Reports, Final Beneficiary
Reports
Source: Human Resource Development Operational Programme, second version
Table 1.3.2 Result indicators on Measure 1.3, Priority axis 1 as found in SAR
Count
6
7
8
Indicators
HRD OP
OIS
Decrease in the rate of
Revised: Percentage
complaints (advice slip)
of decrease in the
on the unregistered
rate of complaints
employment to the
(advice slip) on the
Insurance Inspection
unregistered
Board
employment to the
Social Security
Inspection Board
Increase in the number
of bilateral projects on
promoting registered
employment between
the relevant partners
Increase in the rate of
The rate of
registered people in 43
registered
city centres through the
employees in the 12
bilateral projects and
NUTS II regions.
local partnerships
Baseline
Target
Expected
contribution
2432 individual
advice slips
- 15 % =
2067
3
N.A. (Data will be
collected when
the measure is
implemented)
12
158
66.3 %
66.6 %
1033
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
1.3.1 Result Indicator no. 6
1.
Is the indicator understandable and measurable from common practice?
“Decrease in the rate of complaints (advice slip) on the unregistered employment to the Insurance
Inspection Board” is not common practice as an indicator. Nevertheless, and as in the first 5
indicators, we should neglect the word “decrease” which has nothing to do in an indicator.
The advice slip, or rather complaints, is reporting on irregular (informal) work. A decrease in number of
“complaints” should, according to the stakeholders, indicate that fewer undertake irregular work.
And then we have to understand the point in “rate”, which again is based upon pure guess work. Well,
it simply does not make sense, not even after looking at the baseline. The target measures a
decrease, which is a very basic analytic process, and nothing valuable. So, no, rate does not make
sense at all.
That leaves us with a raw indicator saying “number of complaints (advice slip) on the unregistered
employment to the Insurance Inspection Board”. The Insurance Inspection Board does not exist
anymore; the complaints are now received by telephone through the hotline, through inspection, etc.
As mentioned the indicator is not common international practice. Data should be available, though the
TAT has received no reply to our requests as mentioned several times before.
190
2.
Is the indicator an Outcome or an Impact Indicator?
The reports of informal activities appear to be an Outcome Indicator, since it mirrors activities for the
complete target group.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
As explained to the TAT the number of complaints should be an indicator of irregular activities.
However, the TAT is not entirely convinced that the indicator precisely measures the magnitude of
irregular activities, and it tends to be plausible to suggest that number of complaints easily could vary
according to other factors than the activities related to the HRD OP, including perhaps economic
development and general wage increases in the formal labour market.
Of course, economic development and general wage increases in the formal labour market could be
seen as being a benefit for the society, but does a decrease in “number of complaints on irregularities”
due to these factors actually measure the magnitude of irregularities, and could we hope that
economic development and general wage increases in the formal labour market would stop
irregularities? Presumably not, of we should trust international experience and theory of informal
sectors.
And that leads us to a reservation about the doubt on the ability to reflect a relevant and sustainable
process and progress through the indicator. It is not perfectly clear to the TAT that this measure by
itself is sustainable; will it work when times get tough and the economic crisis hits the NUTS 2
Regions?
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
The relevance of the indicator subject to the Priority, the measure, the specific objectives, and the
social and economic conditions facing the NUTS 2 Regions of HRD OP, is studied in the following
table which, as above, summarizes the framework of the indicator (Priority, Measures, and Objectives)
and tries to provide some brief comments to each of the elements within.
Framework
1) Attract and retain more people in
employment
2) Increase labour force participation
(of women)
3) Decrease unemployment rates
(especially for young people)
4) Promote registered employment
Relevance of the Indicator
No. The indicator do not address issues of attracting
and retaining employment; it simply measure
irregularities
No. Complaints changes nothing; it is doubtful that
the employer will hire the same if it was formal and
registered
No
No. The indicator tells nothing about registered
employment; it just measure irregularities
Check




The indicator might tell us about the magnitude of the problem, and accordingly the cause of action,
but it will not be able to measure progress towards our priorities, measures, and objectives.
Using a negative or reciprocal indicator of the development we wish to achieve will always lead to this
conclusion. It does not disqualify the indicator per se.
191
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
According to SSI data is very available. As mentioned the TAT have had no means to test the quality
of the data, and we are not able to draw upon international practice.
6.
Overall conclusion and preliminary recommendation
It is not easy to draw up conclusions on the “number of complaints on irregularities related to work” as
a viable Outcome Indicator for a programme which serves to promote registered employment. The
indicator is presumably established as a measure of the magnitude of the problem, to which
measurement the TAT still lacks proof of creditability; at least it is not a standard way to measure the
size of the informal sector.
Of course, it is always nice when citizens stop making complaints, but the TAT is not convinced that a
stop in the number of complaints should be received as a proof of a problem which no longer exists.
Similarly, the TAT is not at all convinced that a decrease in the number of complaints received should
be taken as a proof of a decreasing scale of the informal sector. At least, that TAT remains to be
convinced thereof.
Accordingly, the Interim Evaluation is in no position to make recommendations. If the stakeholders fullheartedly believe in the measure the Interim Evaluation has no objection.
1.3.2 Result Indicator no. 7
1.
Is the indicator understandable and measurable from common practice?
Leaving out the term “Increase”, the “number of bilateral projects on promoting registered employment
between the relevant partners” appears fairly straight insofar we know who the relevant partners are.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is unfortunately neither an Outcome not an Impact Indicator. The indicator measure
output of certain initiatives.
1.3.3 Result Indicator no. 8
1.
Is the indicator understandable and measurable from common practice?
Again we have to omit the word “Increase”. The term “rate” in “rate of registered people” is not
explained. We would presume that we are talking about employment rate, which makes sense.
Accordingly, the indicator reads “the employment rate among registered people in 43 city centres
through the bilateral projects and local partnerships”. The words has been changed in the OIS into
“the employment rate in the 12 NUTS II regions”, which is slightly more sharp, since “employees”
appear to be more in line with the overall measure than “people”. Furthermore, the revision has
omitted the “… through the bilateral projects and local partnerships” which has made the indicator far
stronger.
2.
Is the indicator an Outcome or an Impact Indicator?
The employment rate in the 12 NUTS II regions is without any reasonable doubt an Outcome
Indicator.
192
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Employment rates are a very commonly used Outcome Indicator and have proven to be solid and
transparent. When it comes to documentation of a sustainable process under the heading of
promoting registered employment it is the most obvious.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
We do not need a table here: Yes, without any doubt the indicator is highly relevant, perhaps the most
relevant given the measure.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
Data is very available and solidly supported by TÜİK / TURKSTAT which makes it even far more
viable. Given that the indicator is highly transparent and available for all NUTS 2 regions, not just in
Turkey but in all EU Member States, the indicator can be used in international benchmark as well
ensuring a solid evaluation in due time.
The data is used in the Socio-Economic background analysis of Chapter 3.
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation gives credit to the use of this indicator. Further comments will follow in later
sections of this chapter.
1.3.4 Feedback from survey on indicators among the stakeholders on Result Indicator 68
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
21.3%
13
8.2%
5
8.2%
5
4.9%
9.8%
3
6
13.1%
8
8.2%
0.0%
11.5%
9.8%
4.9%
5
0
7
6
3
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.1.4 Measure 1.4
To improve public employment services
Specific Objective:
•
To improve the quality and efficiency of public employment services especially by
strengthening administrative capacity.
193
Table 1.4.1 Result indicators on Measure 1.4, Priority axis 1
Count
9
10
Indicator
Baseline (2006)
Increase in the number of
unemployed people counselled
18654
Increase in the placement
rates of İŞKUR
22.3 %
Target
(2009)
Target
(2011)
M:
11000
F:
11000
30%
M:12500
F: 12500
İŞKUR Records, MoLSS
Programme Reports,
Final Beneficiary Reports
30%
Surveys, Polls, MoLSS
Programme Reports,
Final Beneficiary Reports
Source of Verification
Source: Human Resource Development Operational Programme, second version
Table 1.4.2 Result indicators on Measure 1.4, Priority axis 1 as found in SAR
Count
9
11
12
10
Indicators
HRD OP
OIS
Increase in the number
This indicator has
of unemployed people
been turned into an
counselled
Output Indicator
Revised: Increase in
the number of
qualified reports,
policy documents
and actions
produced by the
Provincial
Employment and
Vocational Training
Number of
communication
campaigns organized
for increasing the
visibility of İŞKUR
Revised: Increase in
the percentage of the
registration rate to
İŞKUR
Increase in the
placement rates of
İŞKUR
Baseline
Target
NA (Data will be
collected when
the measure
implemented)
43
22.3%
30%
Expected
contribution
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
The original Indicator of “number of unemployed people counselled” has been deleted from the pool of
Result Indicators and entered pool of Output indicators in the OIS; quite justified. Three new indicators
have been added, of which the first will be referred to as “Result Indicator Number 9” in the following.
1.4.1 Result Indicator no. 9
1.
Is the indicator understandable and measurable from common practice?
Again omitting the use of the term “Increase” the indicator reads: “the number of qualified reports,
policy documents and actions produced by the Provincial Employment and Vocational Training”. The
indicator is understood with ease and presumably easy to measure.
194
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is so obviously an Output Indicator. It might not be related to the HRD OP exclusively,
and the institution might not be related to the public employment service, but that does not make the
indicator less an Output indicator.
1.4.2 Result Indicator no. 10
1.
Is the indicator understandable and measurable from common practice?
Omitting the term “Increase” the indicator is “placement rates of İŞKUR”. The indicator should be
measurable insofar data from İŞKUR is available66.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is an Outcome indicator in this case since the target group is the public employment
service (İŞKUR) and since the measure is to improve the efficiency of İŞKUR.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Well, a relevant and sustainable process can sometimes be tricky to value, especially when we speak
of public service institutions; in present years and in many countries public service has a negative
value. However, from an objective point of view there is no doubt that the indicator reflects a relevant
and sustainable process, mainly justified by the measure itself which supports the capacity building of
İŞKUR.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
The Outcome indicator appears to be relevant mainly since it so closely reflects the measure. There is
no need for a table to prove that.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
Data are available as noted. The data is consolidated by İŞKUR, but the data are based upon
administrative records. As noted above on data generated from administrative records, the general
rule of thumb is that the more we move towards Outcome and not least Impact assessment the
requirement to the statistics becomes of paramount importance. In general we only accept official
statistics for measures of Impact. With respect to Outcome we can use specially designated surveys
and perhaps administrative records, where the administrative records are able to feed in on official
statistics (qualitative) or otherwise be used as control since administrative records are NOT validated
(in terms of terminology we can hardly call administrative record “statistics”, though that word is being
used widely in some central administrations).
In this case we use this comment as a reservation to the use of the indicator; the Interim Evaluation
strongly recommends making sure that the data on placement rates are used with care.
66
Which they are indeed
195
6.
Overall conclusion and preliminary recommendation
The last comment serves as a general conclusion and recommendation. The Interim Evaluation has
no further comments to the use of the indicator.
1.4.3 Result Indicator no. 11
1.
Is the indicator understandable and measurable from common practice?
“Number of communication campaigns organized for increasing the visibility of İŞKUR” is indeed
understandable.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is by all means an Output Indicator since the indicator measure output from public
production.
1.4.4 Result Indicator no. 12
1.
Is the indicator understandable and measurable from common practice?
The 12th Result Indicator reads “Increase in the percentage of the registration rate to İŞKUR”. Again
the use of “Increase” has to be deleted, but how should we understand an “increase in the percentage
of the registration rate”? It is simply impossible to understand.
We will for the remaining part of the study presume that the author means “number of registered
unemployed”.
2.
Is the indicator an Outcome or an Impact Indicator?
“Number of registered unemployed” can easily go for an Outcome Indicator. In the case of Turkey and
in the case of the HRD OP the Interim Evaluation would even say that it is a solid Outcome Indicator.
The reason will follow shortly.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Assuming that the indicator is understood correctly as “number of registered unemployed” the first
point would be a question: Is that a new ‘negative’ or ‘reciprocal’ indicator 67?
Given the circumstances which Turkey faces and acknowledging the challenges facing the
stakeholders to bring the NUTS 2 Regions up to pace institutionally and not least in welfare terms, the
Interim Evaluation agrees with the indicator. But more so, and especially noting the volume of the
informal sector and the challenges facing Turkey in the pathway to make better and more sustainable
jobs, perhaps even jobs which obey the gradually more intensified progress towards OSH regulation,
the Interim Evaluation Report can see the indicator as a token of progress.
Accordingly, the Interim Evaluation agrees that the indicator reflects a sustainable process.
67
Not that it disqualifies an indicator; it just has to be taken into account in the evaluation of the strength of the
indicator not least in terms of relevance for the objectives, priorities, etc.
196
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
Since the measure is to improve the quality of public employment services it appears quite obvious
that the indicator is relevant, not least noting the comments above about the challenges facing Turkey.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
Indeed data are available and used in the Socio-Economic Study of chapter 3. However, the data is
based upon administrative records which reinforce the reservation noted above under Indicator 10.
The data from İŞKUR cannot be used for international benchmarking68.
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation agrees with indicator conditional that the Interim Evaluation has understood the
indicator. The Interim Evaluation issues a light reservation to the use of administrative records, but
sees no doable alternative in this case.
1.4.5 Feedback from survey on indicators among the stakeholders on Result Indicator 910
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
16.3%
7
14.0%
6
4.7%
2
4.7%
4.7%
2
2
11.6%
5
14.0%
4.7%
7.0%
14.0%
4.7%
6
2
3
6
2
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.2 Priority axis 2
To enhance investment in human capital by increasing the quality of education, improving the
linkage between education and labour market, and raising enrolment rates at all levels of
education, especially for girls
III.2.1 Measure 2.1
To increase enrolment rates particularly for girls at all levels of education and vocational
training
68
Reservation should further be made to cross-regional benchmarking due to the uncertainty in administrative
records. The Interim Evaluation is sure that İŞKUR does every possible effort to ensure valid data through QA
measure within its organisation, but so does every country and still errors are very visible. The point is that errors
occur and become highly visible notably through cross-section analyses, perhaps due to light divergences in local
practices in registration.
197
Specific Objectives:
•
To increase enrolment rates particularly for girls in secondary education and VET
•
To decrease drop-outs particularly for girls
•
To increase vocational skills and competences of labour force, particularly of girls and
women
•
To promote awareness-raising for parents on importance of education, especially for girls.
Table 2.1.1 Result indicators on Measure 2.1, Priority axis 1
Count
13
14
15
Indicator
Baseline (2006)
Target
(2009)
Target
(2011)
Increased percentage of girls
accessing to education
particularly secondary
education as a result of
programme intervention.
Reduction in the number of
girls dropping out of secondary
education and VET
77 %
80-85 %
85%
School enrolment rates
MoNE statistics, MoLSS
Programme Reports,
Final Beneficiary Reports
NA(It will be
determined after
the measure is
implemented)
NA (It will be
determined after
the measure is
implemented)
15 %
reduction
17%
15 %
increase
17%
School enrolment rates,
MoNE statistics, MoLSS
Programme Reports,
Final Beneficiary Reports
School enrolment rates,
MoNE statistics, MoLSS
Programme Reports,
Final Beneficiary Reports
Increase in the number of girls
graduating from secondary
education and VET
Source of Verification
Source: Human Resource Development Operational Programme, second version
Table 2.1.2 Result indicators on Measure 1.1, Priority axis 1 as found in SAR
Count
13
14
15
Indicators
HRD OP
OIS
Increased
Increase in the
percentage of girls
access rate of
accessing to
education,
education
particularly
particularly
secondary
secondary education education for girls
as a result of
programme
intervention.
Reduction in the
Decrease in girls'
number of girls
dropout rates at
dropping out of
secondary
secondary education education and
and VET
VET schools
Increase in the
number of girls
graduating from
secondary education
and VET
Baseline
Target
Expected
contribution
491
77%
Increased
percentage: 8085% 3- 8%
increaseOutput
indicator: 50 000
girls
NA (It will be
determined after
the measure is
implemented)
15% reduction
No accessible
data yet
NA (It will be
determined after
the measure is
implemented)
15% increase
No accessible
data yet
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
Table 2.1.3 Output indicators on Measure 1.1, Priority axis 1 as found in MIS
Ref.
13
14
15
Indicator
Increased percentage of girls accessing to
education particularly secondary education as a
result of programme intervention.
Reduction in the number of girls dropping out of
secondary education and VET
Increase in the number of girls graduating from
secondary education and VET
Target
Achieved
12,808
1,880
4,878
-240
9,133
-14
Source: Monitoring Information System, Operating Structure, MoLSS
198
2.1.1 Result Indicator no. 13
1.
Is the indicator understandable and measurable from common practice?
Indicator 13 reads “Increased percentage of girls accessing to education particularly secondary
education as a result of programme intervention”. The percentage of girls accessing to education is
not understood, since we (again) lack a denominator. We shall assume that the author might have
meant “number of girls accessing to education particularly secondary education as a result of
programme intervention relative to total girls in the same age group”.
Nevertheless, the indicator has been revised in the OIS into “Increase in the access rate of education,
particularly secondary education for girls”. We shall presume that access means enrolment, which
simply transform the indicator into something which in common use is titled “enrolment rates”.
Accordingly: “enrolment rates of girls with special focus on secondary education levels”.
2.
Is the indicator an Outcome or an Impact Indicator?
It is generally not normal to have a measure, objective or a priority which is named after an indicator,
but we will leave that out of here. Yes, the indicator is by all means a well-known Outcome Indicator.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Certainly, the indicator reflects a relevant and sustainable process which is of high priority of the HRD
OP.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
Yes, the indicator is very relevant for the HRD OP and the challenges facing the NUTS 2 Regions.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
Data is available from TÜİK / TURKSTAT ensuring a solid platform for evaluation.
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation has no further comments.
2.1.2 Result Indicator no. 14
1.
Is the indicator understandable and measurable from common practice?
Indicator 13 reads “Reduction in the number of girls dropping out of secondary education and VET”
later in the OIS changed to “Decrease in girls' dropout rates at secondary education and VET
schools”. Omitting the “decrease-thing” the indicator is: “Dropout rates for girls at secondary education
and VET schools”. That is a well-known indicator.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is an Outcome Indicator by nature.
199
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
The indicator reflects a relevant and sustainable process and addresses a general problem throughout
the world.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
Indeed, the indicator is relevant for the overall purpose of HRD OP as well as the challenges facing
the NUTS 2 Regions.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
The TAT has not obtained any data on the subject. The data is commonly based upon administrative
records which generate a general warning. Otherwise, data should be within reach with some effort
from Ministry of National Education.
6.
Overall conclusion and preliminary recommendation
Making the reservation of the quality of the data based upon administrative records, mainly
administrative records originating from single and more or less autonomous institutions with
presumably divergent approaches to QA measures, the Interim Evaluation has no additional
comments to the use of the Outcome Indicator.
2.1.3 Result Indicator no. 15
1.
Is the indicator understandable and measurable from common practice?
Indicator 13 reads “Increase in the number of girls graduating from secondary education and VET”
which transformed into something readable and doable becomes: “number of girls graduated from
secondary education and VET”. Again, the indicator is well-known and commonly used.
2.
Is the indicator an Outcome or an Impact Indicator?
Also this indicator is an Outcome Indicator by nature.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
The indicator is relevant for the overall process of the HRD OP.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
Indeed, the indicator is relevant for the overall purpose of HRD OP as well as the challenges facing
the NUTS 2 Regions.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
Data are available from TÜİK/TURKSTAT.
200
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation has no comments to the use of the Outcome Indicator.
2.1.4 Feedback from survey on indicators among the stakeholders on Result Indicator
13-15
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by
submission of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or
regions)
Data is only generated from on-going projects (we cannot see what status are in
locations without projects, before the project is launched, and after the project
has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
30.4%
7
4.3%
1
13.0%
3
8.7%
2
4.3%
1
13.0%
3
4.3%
0.0%
0.0%
13.0%
8.7%
1
0
0
3
2
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.2.2 Measure 2.2
To improve the content and quality of Vocational and Technical Education
Specific Objectives:
•
To increase the quality of vocational and technical education and training addressing
curriculum, teachers, learning environment and school managers
•
To increase flexibility of the education system by facilitating vertical and horizontal
transitions between different types of programmes in the VET system
•
To make VET an attractive option for students
•
To promote cooperation between social partners, schools and VET centres and private
sector
Table 2.2.1 Result indicators on Measure 2.2, Priority axis 1
Count
16
17
Target (2009)
Target (2011)
Source of
Verification
Indicator
Baseline (2006)
The level of satisfaction
of teachers, students
and entrepreneurs who
participated in the
activities regarding the
content and quality of
Vocational and
Technical Education.
Increasing demands for
45 pilot vocational
schools.
N.A.(Baseline
50 %
data will be
determined when
the measure
implemented)
70%
Surveys, MoLSS
Programme
Reports, Final
Beneficiary
Reports
N.A. (It is not
possible to
measure it at the
moment)
Min. 20 %
increase in
enrolment for each
individual
supported
Student preregistration
records, MoLSS
Programme
Reports, Final
Min. 20-25 %
increase in
enrolment for each
individual
supported
201
Count
18
Indicator
Teacher training in line
with the newly updated
curriculum.
Baseline (2006)
Target (2009)
N.A.
vocational school
(demands for 45
pilot vocational
schools will be
increased to 20-25
% after the
implementation of
the project in 45
pilot institutions)
1500 Teachers
Target (2011)
vocational school
(demands for 45
pilot vocational
schools will be
increased to 18 %
after the
implementation of
the project in 45
pilot institutions)
4000 Teachers
Source of
Verification
Beneficiary
Reports
MoLSS
Programme
Reports, Final
Beneficiary
Reports
Source: Human Resource Development Operational Programme, second version
Table 2.2.2 Result indicators on Measure 1.1, Priority axis 1 as found in SAR
Count
16
Indicators
HRD OP
OIS
The level of
Increased
satisfaction of
percentage of the
teachers, students
teachers, students
and entrepreneurs
and entrepreneurs
who participated in
satisfied with the
the activities
content and
regarding the content quality of
and quality of
Vocational and
Vocational and
Technical
Technical Education.
Education.
Increasing demands
for 45 pilot vocational
schools.
Baseline
Target
N.A.(Baseline
data will be
determined when
the measure
implemented)
50 % (satisfaction of
teachers, students
and entrepreneurs will
be increased to 50 %
after the
implementation of
measure in 45 pilot
institutions)
N.A. (It is not
possible to
measure it at the
moment)
Min. 20-25 %
increase in enrolment
for each individual
supported vocational
school (demands for
45 pilot vocational
schools will be
increased to 20-25 %
after the
implementation of the
project in 45 pilot
institutions)
17
18
Teacher training in
line with the newly
updated curriculum.
Expected
contribution
The indicator is not mentioned in the SAR
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
2.2.1 Result Indicator no. 16
1.
Is the indicator understandable and measurable from common practice?
Indicator 16 reads “the level of satisfaction of teachers, students and entrepreneurs who participated in
the activities regarding the content and quality of Vocational and Technical Education” later in the OIS
revised to read: “Increased percentage of the teachers, students and entrepreneurs satisfied with the
content and quality of Vocational and Technical Education”.
Omitting “increase” and changing the word “percentage” into the more correct term “share”, the
indicator is “the share of teachers, students and entrepreneurs satisfied with the content and quality of
Vocational and Technical Education”.
202
The indicator calls for designated surveys. “Satisfaction” is a difficult measure to handle but the
method is widely used. From a more theoretical standpoint it tends to become non-sense to use the
indicator since there will be no baseline, and since satisfaction is extremely subjective and not
explicitly specified from a qualitative point of view. But that discussion is not designed for an Interim
Evaluation Report.
2.
Is the indicator an Outcome or an Impact Indicator?
“Satisfaction” as an indicator is still undergoing theoretical discussions on where to place it; Output or
Outcome. For the sake of the purpose of this very report the Interim Evaluation will accept the
indicator as an Outcome Indicator.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Yes, the indicator, if done properly, reflects a relevant and hopefully sustainable process toward
gradually better, relevant and market-oriented trainings.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
The indicator is highly relevant subject to the Priority and its measures.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
Apparently, no data are available yet, but presumable the proper means has been established to
ensure data.
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation has some general theoretical reservation towards the indicator, but accepts is
relevance. The Interim Evaluation has no further comments.
2.2.2 Result Indicator no. 17
1.
Is the indicator understandable and measurable from common practice?
Indicator 17, “Increasing demands for 45 pilot vocational schools” is not at all understood. The word
“Demand” has to be specified and transformed into something measurable.
From the description of the target it appears that the stakeholders want to measure demand as
“increase enrolment” which is pure non-sense; enrolment is not a unified measure of pure demand,
since various factors, including institutional, structural / economic, and infra-structural factors, have
heavily impact on students’ choice of education.
The point is, of course, that demand measure voluntary behaviour. In order to use enrolment rates as
a measure of demand, the measure has to be cleaned from all factors which are not of behavioural
origin. That task might be impractical perhaps even impossible.
Another way to measure demand is to establish surveys directed towards those pupils / students who
consider their choice of education.
203
In general, the Interim Evaluation is uncertain of the meaning of the indicator and cannot find the way
it is supposed to be performed. For the sake of the report the Interim Evaluation presume that the
indicator reads “enrolment rates in vocational schools”.
In that case the indicator is the same as Indicator 13.
2.2.3 Result Indicator no. 18
1.
Is the indicator understandable and measurable from common practice?
Indicator 18 reads: “Teacher training in line with the newly updated curriculum”. The indicator is not
mentioned in the SAR.
The Interim Evaluation does simply not understand the indicator at all. Presumably it is established to
measure the “number of teachers trained in newly developed curriculum”. In that case the indicator is
an Output Indicator by nature.
2.2.4 Feedback from survey on indicators among the stakeholders on Result Indicator
16-18
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
11.4%
4
8.6%
3
5.7%
2
2.9%
8.6%
1
3
8.6%
3
11.4%
8.6%
8.6%
17.1%
8.6%
4
3
3
6
3
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.3 Priority axis 3
To increase adaptability of workers, enterprises and entrepreneurs, in particular by promoting
lifelong learning and encouraging investment in human resources by enterprises and workers
III.3.1 Measure 3.1
Promoting the development and implementation of coherent and comprehensive strategies for
lifelong learning
Specific Objectives:
•
To provide basic skills and competences especially for the disadvantaged and women
•
To improve quality of non-formal training in line with the LLL perspective
•
To develop a comprehensive new approach to valuing learning to build bridges between
different learning context and learning forms and to facilitate access to individual
pathways of learning.
204
•
•
•
•
To strengthen the existing VET institutions for organizing VET courses in line with LLL
strategies and according to the ECVET principles.
To support establishment of Vocational Standards Development, Testing and Certification
Centres (VOC-TEST) as accredited bodies of vocational standard development, testing
and certification under the monitoring and supervision of Vocational Qualifications
Authority.
To support the establishment of National Qualification System
To promote the linkage between LLL, education system and labour market
Table 3.1.1 Result indicators on Measure 3.1, Priority axis 3
Count
19
20
21
22
Target
(2009)
Target
(2011)
Indicator
Baseline (2006)
Share of primary education
and secondary education
graduates among participants
certified and tested in terms
of vocational knowledge and
skills in 10 VOC-TEST
Centres
30 % to be
graduates of
primary
education;
70 % to be
graduates of
secondary
education
313.584 (2005
10-15 %
data from Public increase
Education
Centres)
30 % to be
graduates of
primary
education;
70 % to be
graduates of
secondary
education
16-20%
Increased percentage of
participants who obtained a
certificate in basic skills
84.662 (2005
15-20 %
data from Public increase
Education
Centres)
16-20%
Increased percentage of
those who found work after
completing an employability
course
NA
16-18%
Increased percentage of
participants who obtained a
certificate in vocational
knowledge and skills
10-15 %
Source of Verification
Enrolment statistics,
Attendance rates,
MoLSS Programme
Reports, Final
Beneficiary Reports
Enrolment statistics,
Attendance rates,
MoLSS Programme
Reports, Final
Beneficiary Reports
Enrolment statistics,
Attendance rates,
MoLSS Programme
Reports, Final
Beneficiary Reports
Enrolment statistics,
Attendance rates,
MoLSS Programme
Reports, Final
Beneficiary Reports
Source: Human Resource Development Operational Programme, second version
Table 3.1.2 Result indicators on Measure 3.1, Priority axis 3 as found in SAR
Count
19
20
Indicators
HRD OP
Share of primary
education and
secondary
education graduates
among participants
certified and tested
in terms of
vocational
knowledge and
skills in 10 VOCTEST Centres
Increased
percentage of
participants who
obtained a
certificate in
vocational
Baseline
Target
OIS
30 % to be
graduates of
primary
education; 70 %
to be graduates of
secondary
education
313.584 (2005
data from Public
Education
Centres)
10-15 % increase
Expected
contribution
190
384 (LLL) + 2977
(PWE) + 3068
(PYE)
App. (-) 20000
205
Count
21
22
Indicators
knowledge and
skills
Increased
percentage of
participants who
obtained a
certificate in basic
skills
Increased
percentage of those
who found work
after completing an
employability course
Baseline
Target
Expected
contribution
84.662 (2005
data from Public
Education
Centres)
15-20 % increase
962+120:1082
App. (-) 15000
NA
10- 15%
%63 (194/306)
(LLL)
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
Table 3.1.3 Output indicators on Measure 3.1, Priority axis 3 as found in MIS
Ref.
Indicator
19
20
21
22
Share of primary education and secondary education
graduates among participants certified and tested in terms
of vocational knowledge and skills in 10 VOC-TEST
Centres
Increased percentage of participants who obtained a
certificate in vocational knowledge and skills
Increased percentage of participants who obtained a
certificate in basic skills
Increased percentage of those who found work after
completing an employability course
Target
Achieved
418
158
2,178
1,243
2,100
1,718
575
209
Source: Monitoring Information System, Operating Structure, MoLSS
3.1.1 Result Indicator no. 19
1.
Is the indicator understandable and measurable from common practice?
Indicator 19 reads “Share of primary education and secondary education graduates among
participants certified and tested in terms of vocational knowledge and skills in 10 VOC-TEST Centres”.
The indicator is understood though the phrasing could need a hand.
2.
Is the indicator an Outcome or an Impact Indicator?
It is not crystal clear to the TAT what the specific purpose of the related specific objective is: “To
support establishment of Vocational Standards Development, Testing and Certification Centres (VOCTEST) as accredited bodies of vocational standard development, testing and certification under the
monitoring and supervision of Vocational Qualifications Authority”. And that fact makes is slightly
difficult to understand the indicator.
If the purpose is to “support” the VOC-TEST centres forcing primary education and secondary
education graduates to enter the school might soon appear to be for the benefit of establishing the
VOC-TEST Centres and not the in the pursuit of promoting Lifelong Learning as such. And in that case
the indicator appears to be an Output Indicator.
In other words, the Interim Evaluation is in doubt of the relevance of the specific object as well as the
indicator. However, for the sake of the report the Interim Evaluation ignores this light reservation.
206
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Since the Interim Report ignores its general reservation towards the objective and the indicator, and
instead believes the indicator serves a purpose, the Interim Evaluation can agree that ensuring
certificates for primary education and secondary education graduates in terms of vocational
knowledge and skills could potentially reflects a relevant process.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
Basically, the TAT does not know if the indicator is relevant to the overall purpose of the HRD OP. The
TAT is not convinced at all.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
The TAT has received no data on the indicator, which cannot be given by TÜİK/TURKSTAT. The TAT
is in doubt of the purpose of the indicator’s relevance, and cannot answer the question of the data
quality from a theoretical perspective.
6.
Overall conclusion and preliminary recommendation
Since the Interim Evaluation is uncertain about the benefit of this indicator other far more vital
stakeholders might sense the same. It is highly recommended to ensure a solid knowledge of the
purpose of the indicator and from there take a discussion of the relevance of the indicator. Basically,
the Interim Evaluation sees no point in having the indicator as an Outcome indicator at all.
3.1.2 Result Indicator no. 20
1.
Is the indicator understandable and measurable from common practice?
Indicator 20 reads “Increased percentage of participants who obtained a certificate in vocational
knowledge and skills”. The denominator in “percentage of…” is not understood, and a baseline of
313,584 does simply not give a clue; it appears to be a number and not a percentage.
The Interim Evaluation assumes the correct indicator is “number of participants who obtained a
certificate in vocational knowledge and skills” which has a target which stipulates an increase 69.
2.
Is the indicator an Outcome or an Impact Indicator?
Determination of the difference of Output and Outcome will depend on the definition of the word
“participants”. In case “participants” refer to specific programmes initiated as actions or measure, the
indicator will be an Output indicator. In case the author basically thought of participants who by own
means and effort are engaged in training the indicator will obviously be an Outcome indictor. The
degree of voluntary behaviour is accordingly important.
The Interim Evaluation acknowledges the difficulties in finding the right phrasing and shall
subsequently assume that the intention refers to the latter of the two. Accordingly, the indicator is an
Outcome Indicator.
69
A main source of the problem is the overall measure, obviously. The measure seeks to stimulate Lifelong
Learning which makes it fairly difficult to spot a natural denominator. Percentage of day-students (initial
VET/TVET) would not be reasonable, and the same goes with total students as well as total employees.
207
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
Upgrading the labour force is always relevant.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
The indicator is relevant to the measure and the challenges facing the NUTS 2 Regions for sure.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
The TAT has seen no data. The TAT shall assume that data on number of participants who obtained a
certificate in vocational knowledge and skills is very available or at least the data can easily be
produced.
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation has no comment to the choice of Outcome Indicator but to sharpen the name
of the indicator.
3.1.3 Result Indicator no. 21
1.
Is the indicator understandable and measurable from common practice?
Indicator 21 reads “Increased percentage of participants who obtained a certificate in basic skills”
which generates the same mystery as in indicator 20 above. The denominator is simply not clear.
Reformulating the indicator to “number of participants who obtained a certificate in basic skills” can be
understood70.
The line of arguments follows the points noted under indicator 20 above.
3.1.4 Result Indicator no. 22
1.
Is the indicator understandable and measurable from common practice?
Indicator 22 reads: “Increased percentage of those who found work after completing an employability
course”. Presuming that the denominator is the total participants in employability courses the indicator
is “number of participants from employability course who obtains a job after completion relative to total
participants on the employability courses initiated”.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is an Outcome indicator.
3.
Does the Indicator mirror or reflect a relevant and sustainable process?
The indicator reflects a relevant process of the HRD OP.
4.
Is the Indicator relevant subject to the Priority, the measure, the specific objectives, and
the social and economic conditions facing the NUTS 2 Regions of HRD OP?
Please note the comments to the phrasing “participants” under Indicator 20 above. The problem is not severe in
this indicator since the indicator basically refers to the effect “employment”.
70
208
The Interim Evaluation would have expected to see some sort of qualitative approach to the issue,
which could have been added with a requirement of e.g. “number of participants from employability
course who obtains a job which mirrors the upgraded qualifications gained after completion …”
Otherwise, the indicator appear to be appropriate and in line with its purpose.
5.
Are data available and are the data consolidated ensuring means to do solid evaluation?
The indicator will have to be based upon combining administrative records which generates a general
warning (a reservation rather) as noted several times above. In addition, the indicator might imply a
need for follow-up procedures which can be slightly man-power consuming; at least the administrators
/ training schools have to do tracer studies.
6.
Overall conclusion and preliminary recommendation
The Interim Evaluation has no additional comments to the indicator.
3.1.5 Feedback from survey on indicators among the stakeholders on Result Indicator
19-22
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
21.7%
5
13.0%
3
8.7%
2
0.0%
8.7%
0
2
8.7%
2
8.7%
4.3%
0.0%
17.4%
8.7%
2
1
0
4
2
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.3.2 Measure 3.2
To increase adaptability of employees and employers by investing more in human capital
Specific Objectives:
•
To improve adaptability of employees in SMEs by facilitating their access to training, in
particular for low-skilled workers,
•
To improve adaptability of employers in SMEs by designing and disseminating innovative
and sustainable forms of work organization, which support labour productivity and quality
at work,
•
To increase training capacities of social partners, NGOs, VET Institutions and public
bodies in order to establish capacity to facilitate adaptability of the employees and
employers.
209
Table 3.2.1 Result indicators on Measure 3.2, Priority axis 3
Count
23
24
Indicator
Baseline (2006)
Increased adaptability of
N.A.
workers in terms of having
extra skills or abilities related
to their position (percentage
of participants increasing
their salary or being
promoted or having extra
responsibilities)
Increased adaptability of
N.A.
employers in terms of having
extra skills or abilities in order
to be competitive enough
(percentage of participants
increasing the number of
workers, change of the
branch of activity )
Target
(2009)
Target
(2011)
15 %
15%
Surveys, Records of
Final Beneficiaries
MoLSS Programme
Reports
15 %
15%
Surveys, Records of
Final Beneficiaries
MoLSS Programme
Reports
Source of Verification
Source: Human Resource Development Operational Programme, second version
Table 3.2.2 Result indicators on Measure 3.2, Priority axis 3 as found in SAR
Count
23
24
25
Indicators
HRD OP
OIS
Increased
adaptability of
workers in terms of
having extra skills or
abilities related to
their position
(percentage of
participants
increasing their
salary or being
promoted or having
extra
responsibilities)
Increased
adaptability of
employers in terms
of having extra skills
or abilities in order
to be competitive
enough (percentage
of participants
increasing the
number of workers,
change of the
branch of activity )
Increased
training activities
on the
requirements of
adaptability given
by social
partners to their
members
Baseline
Target
Expected
contribution
NA
15%
NA
15%
NA
20%
Source: Sectoral Annual Report on the Implementation of the Human Resources Development Operational Programme, 2010,
Operating Structure, MoLSS, dated June 2011
210
3.2.1 Result Indicator no. 23
1.
Is the indicator understandable and measurable from common practice?
Indicator 23 reads: “Increased adaptability of workers in terms of having extra skills or abilities related
to their position (percentage of participants increasing their salary or being promoted or having extra
responsibilities)”.
The Interim Evaluation finds it very difficult to understand how the indicator should be measured.
Basically, there are two distinct indicators which are combined, of which second is mentioned in
parenthesis. Both of the two indicators appear to create severe difficulties in measurement. In fact, the
TAT has never found similar practice elsewhere.
The problem of the first part (the first indicator) is to find the workers who have achieved extra skills
and abilities related to their position; it will presumably necessitate solid surveys within a very large
sample of establishments. Secondly, all data has to be coded by ISCO-level as well as by training
courses achieved. As far as the TAT knows Turkey has no database on education and training
programmes by domain, including LLL training programmes. And finally, time is essential; participation
in trainings five or ten years ago does not really count anymore, and yet we would not expect
progression to happen as soon as the training has been concluded.
The problem with the second indicator is related to the last part; it appears quite impossible to obtain
detail data on individual level. In addition, it appears far too narrow to believe that salaries and
promotion is given by upgrade skills alone.
Accordingly, the indicator, however defined, is almost criminal problematic and not least extremely
difficult to evaluate upon if data should be available in a million years.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator is an Outcome indicator insofar the target group of the indicator covers all workers and
not just is directed to a selected number of participants in selected training programmes. The use of
the word “participants” in the second part of the indicator’s name is accordingly problematic. If the
indicator only measures those who has participated (perhaps relative to those who has not
participated) the indicator is an Output indicator.
Since we would believe that the idea of the indicator is to measure the wage level and job functions
within a restricted pool of workers, who has recently participated in a training course, a relate the study
to an average or at least a selected control group, the TAT tend to believe that the indicator is
designed as an output indicator.
3.
Feedback from survey on indicators among the stakeholders on Result Indicator 23
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
Share
N
15.4%
6
15.4%
6
12.8%
5
0.0%
10.3%
0
4
7.7%
3
5.1%
2
211
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
5.1%
5.1%
20.5%
2.6%
2
2
8
1
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
4.
Overall conclusion and preliminary recommendation
Relevant or not, data available or not, the Interim Evaluation does not believe in the indicator for a
number of reasons. The Interim Evaluation recommends deleting the indicator.
3.2.2 Result Indicator no. 24
1.
Is the indicator understandable and measurable from common practice?
Indicator 24 reads: “Increased adaptability of employers in terms of having extra skills or abilities in
order to be competitive enough (percentage of participants increasing the number of workers, change
of the branch of activity)”.
The Interim Evaluation admires the effort of trying to measure “adaptability” both in indicator 23 and
now yet again in indicator 24.
The phrasing in the parenthesis, which presumably was meant to explain the indicator, is not
understood: What is meant by “change of branch of activity” and how can that ever be relevant for
Human Capital Investments? The TAT assumes that “participants increasing the number of workers”
indicate the number of enterprises which experience employment growth after the employer has
participated in a certain training course under the heading of Human Capital Investments.
The indicator is not measurable and even lesser practical. Perhaps a survey can be designed, but it is
highly doubtful that the outcome of it would be valid, significant and interesting.
2.
Is the indicator an Outcome or an Impact Indicator?
The indicator, even if it worked, would be an Outcome Indicator, though somehow only inferior related
to the trainings and the programme. The reason for accepting the indicator as Outcome Indicator is
solely that it measures a potential side-effect arising from certain, named and specific training
programmes for a selected few final beneficiaries. It becomes an Outcome Indicator simply because it
is not a direct Output Indicator; a more correct term would be “induced output”, but that would be too
technical in this context.
3.
Feedback from survey on indicators among the stakeholders on Result Indicator 24
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
Share
N
18.6%
8
11.6%
5
9.3%
4
4.7%
11.6%
7.0%
2
5
3
212
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
2.3%
7.0%
7.0%
16.3%
4.7%
1
3
3
7
2
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
4.
Overall conclusion and preliminary recommendation
The indicator will not work for various reasons. Even if data exist employment growth is hardly ever
correlated with the skills of the employer. And changing branch or economic activity is way out of
bounds in this context.
The Interim Evaluation highly recommends deleting this indicator.
3.2.3 Result Indicator no. 25
1.
Is the indicator understandable and measurable from common practice?
Indicator 25 reads: “Increased training activities on the requirements of adaptability given by social
partners to their members”. The indicator is not mentioned in the HRD OP; it appears only in the OIS.
The TAT understands the indicator and agrees that it might be measurable through surveys, though
far from common practice.
2.
Is the indicator an Outcome or an Impact Indicator?
Again, and as in indicator 24, the indicator, even if it worked, would be an Outcome Indicator simply
because we cannot term it correctly as “induced output”. It measures a side-effect of a given
programme.
3.
Overall conclusion and preliminary recommendation
The Interim Evaluation recommends deleting the indicator.
3.2.4 Recommendations to Result Indicator no. 23-25
Apparently, the Interim Evaluation proposes to delete all the chosen Result Indicators underneath
Measure 3.2 partly since the selected indicators will be highly difficult to measure on objective grounds
and partly since the indicators are designed to measure side-effects which might arise from a billion
other (more relevant) reasons that the participation in a certain training programme.
Of course, the Interim Evaluation cannot leave it with that. The problem with this measure is the
measure itself: “To increase adaptability of employees and employers by investing more in human
capital”. The measure has presumably inspired the stakeholders (the authors) to design impossible
indicators; impossible since the authors have become blind by focusing too narrow on some single
words in a far broader context; “adaptability” and “Human Capital Investment”. Perhaps the authors
could not see that the terms are hardly measurable in quantitative terms.
213
Sometimes we simply have to believe in positive outcome. Sometimes we do not need proof to know
that our effort pays off. But more so, when it comes to measuring the un-measurable other methods
and techniques are called for; techniques which cannot be applied and conducted within a fortnight.
The effect of investment in Human Capital is proven and well-documented in scientific reports. If we
want to proof the effort we do, we have to apply a scientific approach, simply because the topic cannot
be measured from smart techniques alone. It would most presumably take a long range of indicators
to measure a presumed effect (perhaps through advance statistical techniques like e.g. Factor
Analysis), and most presumably the conclusions will be weak (not significant) given the short time the
new paradigms have had an effect.
That fact might well look as an easy solution for a TAT doing an Interim Evaluation, but there is more
to it. The main point is to understand that some topics / measures / ideas / initiatives are simply not
designed to be measured in smart numbers. But that does not necessarily indicate that we lack means
to evaluate our effort; in this case it would be far cleverer to look at long term effects, which basically
calls for Impact indicators.
Impact may arise from various reasons, and we would never link impact indicators directly to single
measures. However, we would expect to see a change, gradually, as time passes on. Sometimes the
changes or the adjustments happen relatively fast allowing us to measure the impact, say, within a
year. Other times, impact takes longer time to show.
In case we want to measure “adaptability” and “Human Capital Investment” we are simply without
means. These terms do not come in terms of numbers and sizes. Not directly at least. And
furthermore, the complete context of the measure is basically a matter of change in behaviour, which
never comes easy and never is done by training courses alone.
In order to keep this section short, the Interim Evaluation would give priority to defining Impact
indicators even though the Impact indicators may work on much broader and general level than the
single measure.
An alternative would be to keep the above mentioned induced output indicators, broaden them up to
mirror a wider group (all employers with a certain economic sector), and then ask the Trade Unions,
the Chambers, and the Employers’ Association to conduct surveys once a year on one single issue:
Contribution to Human Capital Investment (or rather: “How much did your company spend on training
of your employees?”). Obviously, that measure would be the correct Outcome Indicator in this context.
The final recommendation here from the Interim Evaluation would be: Never do the impossible and do
not build monsters, where it is not needed! Monitoring and Evaluation is of paramount importance,
agreed, but no decision maker or stakeholder expects progress reports and evaluation reports to be a
scientific and long-lasting discipline.
Far too often Monitoring and Evaluation consumes far too many resources. Basically, that is a very
poor investment. The stakeholders want answers now. The main point is to design few smart
indicators – it is not to create a monster which no one can comprehend.
III.4 Priority axis 4
To promote an inclusive labour market with opportunities for disadvantaged people, with a
view to their sustainable integration into the labour force and combat all forms of
discrimination in the labour market
214
III.4.1 Measure 4.1
To increase employability of disadvantaged persons, facilitate their access to labour market,
and eliminate barriers for their entrance into labour market
Specific Objectives:
•
To increase employment and labour force participation rate of disadvantaged persons
•
To facilitate disadvantaged persons’ access to labour market
•
To eliminate barriers for disadvantaged persons’ entrance into labour market
Table 4.1.1 Result indicators on Measure 4.1, Priority axis 4
Baseline
Target
Count Indicator
(2006)
(2009)
Increased rate of
90 %
95 %
disadvantaged persons
entered into labour market
(having the qualification
26
and motivation to seek a
job) following the
rehabilitation programmes,
courses and counselling.
Increased rate of
20 % people
30 %
disadvantaged persons
with
people with
employed following the
disabilities
disabilities
27
courses and counselling
13 % ex19.5 % exservices
prisoners
prisoners
Target
(2011)
95%
Source of
Verification
İŞKUR Job Seeking
Records, MoLSS
Programme Reports,
Final Beneficiary
Reports
%35 people İŞKUR Job
with
Placement Records,
disabilities SSI Records, MoLSS
Programme Reports,
%25 exFinal Beneficiary
prisoners
Reports
Source: Human Resource Development Operational Programme, second version
4.1.1 Result Indicator no. 26
1.
Is the indicator understandable and measurable from common practice?
Indicator 26 speaks again of a rate which is not clearly defined. Rephrasing the indicator it becomes:
“the number of disadvantaged persons entered into labour market (having the qualification and
motivation to seek a job) following the rehabilitation programmes, courses and counselling”.
As in indicator 1 above the indicator speaks of “labour market” which we translate to “labour force”,
and as in indicator 1 above the indicator speaks of qualification and motivation which cannot be
measured and is if not further specified.
2.
Is the indicator an Outcome or an Impact Indicator?
Accordingly, the indicator should read: “number of disadvantaged persons entered into labour force
(employed or unemployed) following the rehabilitation programmes, courses and counselling” which is
an Output indicator. The argument is again the same as in Indicator 1.
4.1.2 Result Indicator no. 27
Let us then turn to indicator no. 27, which obviously will receive the same comments as Indicator no. 2
above. Accordingly, “number of disadvantaged persons obtaining a job after participating in
rehabilitation programmes, courses and counselling”.
215
The indicator is again an Output indicator.
4.1.3 Feedback from survey on indicators among the stakeholders on Result Indicator 26
and 27
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
20.0%
4
15.0%
3
15.0%
3
10.0%
5.0%
2
1
10.0%
2
0.0%
5.0%
5.0%
5.0%
10.0%
0
1
1
1
2
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
4.1.4 Recommendation for Indicator 26 and 27
The Interim Evaluation recommends doing the same with the two indicators (indicator 26 and 27) as
above under indicator 1 and 2; to establish an employment rate / labour force participation rate, or at
least obtain an indicator of number of disabled persons entering the labour force.
Of course, we expect that to increase after the commencement of the HRD OP and establishing
rehabilitation programmes, courses and counselling, but we do not specify that (since that turns the
indicators into Output indicators).
III.4.2 Measure 4.2
Better functioning and coordination among the institutions and mechanisms in the field of
labour market and social protection particularly in order to facilitate the integration of
disadvantaged persons into the labour market.
Specific Objectives:
•
To improve the institutional capacities of public service providers to disadvantaged
persons in their employment related services, and the functioning of existing mechanisms,
•
To ensure better coordination among public service providers to disadvantaged people,
including between central and regional/local authorities and in cooperation with NGOs, in
employment related services
Table 4.2.1 Result indicators on Measure 4.2, Priority axis 4
Count
28
Indicator
Baseline (2006)
Increased satisfaction on the
services both in institutional
and target group level (by
staff of institutions and by
disadvantaged persons)
N.A. ( Data will
be collected
when the
measure
implemented)
Target
(2009)
Target
(2011)
25 %
N.A.
Source of Verification
Surveys, MoLSS
Programme Reports,
Final Beneficiary
Reports
Source: Human Resource Development Operational Programme, second version
216
4.2.1 Result Indicator no. 28
The evaluation of indicator 28 follows the same line of arguments as noted under indicator 16 above.
The indicator is an Outcome indicator and relevant. However, the indicator necessitates surveys which
makes it difficult to specify baselines and makes it impractical on a long term scale 71.
The Interim Evaluation has no further comments.
4.2.2 Feedback from survey on indicators among the stakeholders on Result Indicator 28
What is the problem of the indicators under this measure, if any?
Net impact cannot be measured since Impact can arise from other factors in the
environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission
of reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Share
N
13.9%
5
16.7%
6
8.3%
3
2.8%
11.1%
1
4
8.3%
3
0.0%
5.6%
13.9%
19.4%
0.0%
0
2
5
7
0
The stakeholders’ evaluation of the single indicators is given below in section 10.6. The table above indicate the stakeholders
view on where the main problems lie.
III.5 Priority axis 5: Technical Assistance
To improve the coordination, control and implementation of HRD OP and to assist the
Operating Structure and relevant institutions in the further programming, monitoring,
evaluation and implementing tasks.
Specific Objectives
•
Strengthening the necessary capacity at central level to efficiently develop, implement,
evaluate, monitor and control the IPA funds in the period 2007-2011 within the framework
of HRDOP and improving the information about the ESF structures and best examples,
providing assistance in the transition to Decentralised Management without Ex-ante
controls of the EU Delegation.
•
Increasing the absorption capacity of the final beneficiaries and administrative capacity of
stakeholders which may get more responsibilities in the upcoming period.
•
Increasing the information and public awareness with respect to the preparation for and
effective use of the IPA funds in Turkey in line with the HRDOP.
III.5.1 Measure 5.1
Support for Programming, Management, Implementation, Monitoring, Control, Evaluation and
Dissemination Activities
Specific Objectives
•
The main aim of this measure is to provide support to the OS in different steps of the IPA
process and to increase the capacity of the relevant central and local partners in the
related issues.
71
It is doubtful that the surveys will be conducted after the completion of the HRD OP.
217
There is no Result Indicator aligned with the measure.
III.5.2 Measure 5.2
Support for development of absorption capacity of final beneficiaries
Specific Measures
•
The main aim of this measure is to improve absorption capacity of the final beneficiaries in
order to develop a project pipeline in the framework of HRD OP.
There is no Result Indicator aligned with the measure.
III.5.3 Measure 5.3
Information and publicity activities
Specific Objectives
•
The objective of this measure is to assist the implementation of obligations arising from
Article 62 of IPA Implementing Regulation concerning the promotion of the HRDOP and
its operations and informing entities interested in receiving support from the HRDOP, as
well as the general public, about the opportunities provided by the assistance and its
outcomes.
There is no Result Indicator aligned with the measure.
III.5.4 Recommendations to Priority axis 5
The lack of indicators, notably Outcome Indicators, under Priority axis 5 is understandable to some
extent; the complete axis and the underlying measures are devoted to technical assistance which only
works for a temporary period.
Nevertheless, Outcome indicators could be established since the purpose of the technical assistance
is to make capacity building activities for the operation beneficiaries, the Ministries and their subordinates, as well as the operating structure. Obviously, the increase capacity does not only apply to
the management of external aid programmes; hopefully, the main aim of the technical assistance
programmes is to increase the overall capacity within the involved organisations and hereby ensure
better means to carry out daily tasks.
In that case perhaps the specific objectives clings too much to the IPA, though that is understandable;
perhaps the specific objectives makes the stakeholders blind for the long term effect of the technical
assistance and thus blind when it comes to define and establish Outcome Indicators.
The Interim Evaluation Report will propose recommendation to both Outcome and Impact indicators in
a more generic and general section below, partly since both Outcome and Impact indicators should be
based upon a broad perspective where the overall purpose the HRD OP in general should be taken
into account, and partly since the list of indicator within the HRD OP for Turkey is far too long.
However, in case the Operating Structure and the other main beneficiaries find it useful to establish
indicators for each of the priority axes within the HRD OP, Interim Evaluation could propose the
following Outcome Indicators72:
72
The following Outcome Indicators are merely means as inspiration and are not tested
218
•
•
The alignment in the institutional structure with recommendations in JAP, where the target
could be fewer comments in the chapters under negotiation with the EU. The measure
could be ratified by EC documents on the progress of adjustment73;
Satisfaction with the proposed institutional changes recommended, which could be based
upon surveys within the benefitting organisations 74. The indicator could further be
supported by the following:
o The institutional changes followed by recommendations on behalf of the technical
assistance75
PART IV. Reflexions upon the studies in Part I, II and III
IV.1 Findings, main conclusions and recommendations
The Interim Evaluation has been able to recognize some 95 Output indicators from the HRD. In
addition we further investigated 28 Result indicators, which can be divided into the following:
•
9 indicators are Output indicators
•
The remaining 19 indicators are all Outcome Indicators of which:
o 10 Outcome Indicators are workable in practice
o 4 Outcome Indicators could work with some light adjustments
o 5 Outcome Indicators cannot work in its present form and needs to be adjusted
o We found not even one single Impact Indicator
In total we have identified 104 Output Indicators, 19 Outcome Indicators and 0 (zero) Impact Indictors.
Of the 19 Outcome Indicators, ten of them passed the Evaluation without remarks, 9 of the indicators
can be workable with some adjustments. 5 Result Indicators cannot work in practice.
The two tables in the end of this Annex presents the conclusions and recommendations from the study
on both the Output Indicators in Part II as well as the conclusions and recommendations from the
study on Result Indicators in Part III.
Obviously, we are talking about a very high number of indicators, which makes it close to impossible to
undertake evaluation reports; each indicator will have to examined in time series analysis throughout
the implementation and the following years, and each of them has to be examined in various crosssections studies, not just on regional levels but toward target groups and control groups. Using a
conservative estimate based upon experience we could easily suggest that each indicator would
require close to 5 tables, with presumably another 5 tables in annex, and close to 6 pages description.
Due to the number of indicators it would generate evaluation reports of 700 pages, which obviously
would take months to produce. And yet the HRD OP does not even have any Impact Indicators
making it impossible to evaluate the effect of the HRD OP.
Keeping track of a record of 104 output indicators is very appreciable, but no one bother read the
lengthy report the evaluation will create in that case, basically indicating that the system creates a
sequence of reports where only a fraction of the conclusions will be read, understood and acted upon.
In that respect we have to recall that the burden of monitoring and evaluation does not fall on single
M&E experts on central level; the burden is transferred to the complete system, perhaps in some
cases even to final beneficiaries; I.e. finding and creating data, controlling data, ensuring timely input,
ensuring quality procedures in order to produce data of sufficient quality, follow-up, and making
comments.
73
It could also be the number of law packages in alignment with the EU passing through the parliament
Doing surveys within the public sector is generally much easier than market surveys within the private sector
75 Agreed, it would be very difficult to set targets here, since we cannot set a value number of changes and it is
even not certain that it is clever to expect the number to decrease. The measure should be a qualitative measure.
74
219
Forcing discipline through does not make better data and input to the evaluation reports if the burden
obviously is caused by some few central stakeholders’ unreasonable need to know every single detail.
And that comment speaks for some few words on the Monitoring Information System, MIS, which
appears to have turned into a nightmare, at least if we should believe the main stream of comments
received from both central and regional stakeholders. We have added some few comments to the
Monitoring Information System in Chapter 5 above.
IV.1.1 Impact Indicators
The programme lacks Impact Indicators. Without impact indicators it becomes impossible to see the
real effects of the programme. By real effects we mean the impact on the local / regional economy, the
labour market and the welfare.
The Impact indicators will become vital for the final evaluation of the programme. The final evaluation
will require means and tools to do a fair and solid evaluation of the impact of implementing so far 431
projects in 12 NUTS 2 Regions. In order to make the impact assessment data should be secured for a
long time period ensuring solid means to do impact studies across time. And the indicators should be
agreed upon long before the evaluators arrive; it would be impossible for the evaluators to establish
the evaluation criteria and the indicators on top of the impact assessment.
The complete lack of impact indicators tells us that the difference between output, outcome and impact
has never been understood by the stakeholders. But even more it tells us that the stakeholders do not
even know what impact indicators are76.
Let us repeat:
Impact Indicators measures the consequences of the outcomes in terms of wider objectives
(for example literacy rates, health improvement). The definition covers the wider effects of the
outcomes but there might also be higher level impacts, related to broader objectives – growth
and income poverty, for example. In order to turn it into a net impact indicator the effect of
other economic, educational, and employment programmes have to be assessed and
accordingly subtracted.
Let us provide some examples from this HRD OP context.
Example 1. Education. What is Impact here?
Priority axis 2: To enhance investment in human capital by increasing the quality of education,
improving the linkage between education and labour market, and raising enrolment rates at all
levels of education, especially for girls
Measure 2.2: To improve the content and quality of Vocational and Technical Education
Specific Objectives:
o To increase the quality of vocational and technical education and training addressing
curriculum, teachers, learning environment and school managers
o To increase flexibility of the education system by facilitating vertical and horizontal
transitions between different types of programmes in the VET system
o To make VET an attractive option for students
o To promote cooperation between social partners, schools and VET centres and private
sector
76
Otherwise we would have seen at least one impact indicator
220
The following indicators were selected:
1)
The level of satisfaction of teachers, students and entrepreneurs who
participated in the activities regarding the content and quality of Vocational
and Technical Education.
2)
Increasing demands for 45 pilot vocational schools.
3)
Teacher training in line with the newly updated curriculum.
But what is impact of improving education? Is that smiling students? Larger enrolment? Training
of teachers?
The reason for investment in education (for students) is obviously to obtain better job
opportunities. The reason for investing public funds in education is obviously because we
believe it creates larger opportunities to establish a well-developed economy. Accordingly, we
do not invest in education for the sake of teachers, school directors or just in order to park some
teenagers for a period of time: It serves a purpose, that is.
Impact measure the purpose. Here it is obvious to establish an Impact indicator which mirrors
and reflects advancements in education. The most prominent Impact indicator of them all is
subsequently: Productivity.
Other indicators could be:
•
Literacy
•
Health indicators (we shall return to them)
•
GDP / inhabitant and economic growth measures
•
Increased production of advanced goods (some regions experience more or less
a mono-cultural production)
In order to establish Impact indicators we need to look up from the computer and the books and think:
Why do we do this? What do we want to achieve?
The next step in Impact indicators is to find indicators with solid and consistent data. In other words we
need official statistics, hopefully well-documented international harmonized indicators allowing us to
see how much we, Turkey, has improved relatively to the others.
We, the TAT, cannot propose indicators to the HRD OP. That would be madness since proposing
indicators necessitate that you are a real part of the system. The TAT had meetings with some of the
key stakeholders where they were allowed to reflect over the chosen indicators. The meetings were
inconclusive. However, the meetings made clear that the chosen indicators were insufficient and in
some case directly wrong. Most of the comments have been integrated in the study of Result
indicators above.
The meetings would be an obvious place to start a reform of the indicators. Indicators have to make
sense; they should reflect the environment and they should be understood by the real stakeholders.
The table below presents some ideas to pathways for designing the type of indicators which are
needed for larger programmes, including impact indicators. Further well-proven and ready-made
indicators are briefly presented in Appendix to the Study on Indicators in the end of this study.
Finally, the meeting with the DG OSH in the Ministry of Labour made very clear to all, including the
TAT, that the HRD OP seriously lack indicators on OSH and health. It appears obvious to include DG
OSH in future programmes, but also in the revision of the indicator system of the present HRD OP.
221
Table 9.6.1 Example of pathways within education to design of indicators using examples of combined
markers with direction of targets77
INPUT
More spending on
education
OUTPUT



More teachers hired
Better learning
materials
Better schools
(physical)
OUTCOME


More books at libraries
borrowed
More local citizens
educated
IMPACT






General increase in the
standard of education


Increased educational
opportunities/variety of
education


Better and more
advanced learning
materials
National
Qualification
Frameworks
developed
Increased number of
citizens with a wider
variety of skills and
qualification
Less drop-outs


increase in salaries and
wages
more local citizens
qualified for attractive wellpaid jobs

number of unskilled get
semi-skilled or skilled
increased number in higher
education
number of women and men
training and education
increased
quality of the workforce
improves















Improved quality in

educational material and
curricula
Tailor made training
courses for special
target groups


Faster economic growth
Multi-cultural production
Knowledge Society
More visits to movie
theatres
Better living conditions
(income)
higher national living
standard in general
increased gender equality
poverty reduction
longer life expectancy
Increased number of 
citizens with a wider
variety of skills and

qualification
Increased number

educated among
those who previously 
did not finish
education or were
unable to follow the
education
requirement
Less drop-outs
Better standard of the
workforce
A wider/broader
understanding of the
society and on-going
changes
Better standard of the
workforce
A wider/broader
understanding of the
society and on-going
changes









higher productivity
faster and sustainable
growth
fewer and healthier
children
improved family economy
(better housing)
less drugs and alcohol
misuse
less criminality
improved childcare in
families and through
institutions
More citizens with a
globalised and
cosmopolitan orientation
Less conflicts on the labour
market
“Green” economics
Social consciousness
less criminality
higher national living
standard in general
savings on national
finances
increased gender equality
poverty reduction
The mentioned effect is scientifically proven. Please note that “marker” represent themes, and not necessarily
indicators. Please note that the impact markers could fall into every category of inputs
77
222
INPUT
Labour market and

training statistical data
increased and standards
improved
OUTPUT
OUTCOME
Increased number of 
and variety in data
available



improved information about 
the labour market
development

improved information on
labour market needs

improved knowledge about
relevant training needs
nationally and in the
regions
improved conditions for
providing tailor made
training courses
IMPACT
Improved conditions for
political decision making
Better allocation of
financial resources
Loss of money on
irrelevant or less relevant
training courses reduced
IV.1.2 The Multi-level System
The multi-level system is introduced in Chapter 8. The main point is to establish a system of indicators
which feed into each other across the levels. Since the administration of the HRD OP in Turkey
appears to like to know everything it would be an obvious solution.
A note warning is, however, that creating a multi-level system does not make the evaluation easier
and the real danger is to limit time-consuming report-writing in multiple directions. The need to
understand what is necessary to know and what is not essential is accordingly almost reinforced by
creating a multi-level system of indicators, since the multi-level system, where different indicators feed
into different carefully pre-selected monitoring reports, soon can create a new need for still more
reports (now that they are doable).
The objective of the multi-level system is basically quite simple. Different stakeholders within the
system need different information. And above all there are the main stakeholders, perhaps the donors,
who just want to know that the overall programme works in accordance to expectations.
Differentiation, or “who needs what”, is to direct reports to the key stakeholders in accordance to their
interest areas, e.g. the operations they are assigned to administrate and manage.
When the selection has been made, the template for the reports can be designed ensuring an easy
and smooth study and writing process. In most cases, most of the stakeholders in fact agree on a very
slim template based upon the most indicative indicators. And often there is basically no need to create
multi-level reports as well.
The next point is of course to understand and prepare the multi-level system for adjustments.
Adjustments pops up when initiatives (or perhaps projects) exceed the limits and targets, and / or
becomes more or less de-railed subject to their intensions 78. It is in that case the multi-level system
shows its real strengths; the point is that in case we want to know where the problem lies we need
data, and in case the system has data ready at hand, the adjustment can be made promptly. That is
obviously far better than being forced to call in investigators to dig through layers of reason. However,
an early warning system like this comes with a fairly high cost.
In the context of the HRD OP it would be plausible to suggest the following levels:
1. Overall general level with indicators on the overall impact of the complete programmes (all
four types of indicators)
78
Recall that a too positive and strong effect can be just as problematic as weak and no effect at all.
223
2. Operations level; separate report systems for each Priority axis (input, output and outcome
indicators)
3. Project or micro-level (mainly input and output indicators)
Each level should have a very limited number of indicators. Furthermore, it is of vital importance that
most of the indicators are inter-linked if not directly the same, reducing the overall number of
indicators. And then finally, it is important to agree on the reporting standards, both in terms of data
input and reporting. In most cases on subordinate levels simple progress reports (extracts of tables
with few comments) should be sufficient. But it is of vital importance that data is available at the time
for reporting.
IV.1.3 Main Conclusions and recommendations
In the Sectoral Annual Report on the Implementation of the Human Resources Development
Operational Programme, 2010, the following comments to the functionality of the indicators were
found:
“…some difficulties were encountered in determining the actual situation in terms of
achievement of indicators. Although there were some problems stemming mainly from the fact
that practice of monitoring the indicators was at its initial stage, these kinds of preliminary
studies were beneficial from many aspects; for example, importance of entering the
“performance indicators” were emphasized by the MEU and the RGMTT Experts, and the
grant beneficiaries were requested to enter the targets and status of achievement in terms of
indicators determined within their project. In addition, some revisions were made in the
indicators in terms of expression and wording, also some indicators were divided in different
parts for converting the indicator into a measurable format.”
[SAR, p. 38]
Somehow, the Interim Evaluation finds the adjustments insufficient. At least the first and perhaps
prime recommendation is…:
… to revise the current set of indicators, which revision should bring the number of indicators
significantly down. The need for a reform is urgent since the current system of indicators are
insufficient, inappropriate and yet far too ambitious. The Interim Evaluation proposes to bring
the total number of indicators down to 18 indicators: six Output, six Outcome, and six Impact
Indicators79.
I.e. the current system is:
•
Insufficient since the current set of indicators lack Impact measures
•
Inappropriate since many indicators have no doable baselines, lack data, lack
infrastructure, or are just not measuring the right (defined wrongly)
•
Too ambitious due to the number of indicators
The Interim Evaluation has of course suggestions to how ease the administrative burden of a far too
ambitious monitoring and evaluation programme. The suggestions fall in three main categories:
4.
5.
Revise the indictors toughly until a decent and doable target is reached 80; Understand the
basics in evaluation and design a list of indicators which summarizes the main and basic
idea and purpose of the overall programme;
Establish a multi-level system of indicators, ensuring that the main evaluation reports only
contains a very small number of smart indicators. The multi-level system does not
decrease the overall workload of the complete system, but at least the main work falls on
the central level;
79
The mix may vary slightly, of course
The study of indicators calls for a complete reform of the indicators. Perhaps it would be better to start from
scratch.
80
224
6.
Establish solid training programmes in monitoring and evaluation, data control, and quality
assurance, not least on regional level. Place ownership and delegate task where the
single stakeholders can see the purpose.
The first suggestion is surely the hardest one. On the other hand the TAT has been informed that
many central level key stakeholders have never seen the present set of indicators 81. And besides, it is
not just a matter of a far too long list of indicators; the complete programme has no means to make
evaluations due to the complete lack of impact indicators.
The second solution, which draws upon the introduction in Chapter 7, is further explained in a section
below.
The third suggestion leads to open up for a far more flexible administration where it perhaps would be
fair to delegate monitoring and even parts of the evaluation to other stakeholders than the central level
Operation Structure. Agreed, it is exactly what the stakeholders try to achieve, but many of the
stakeholders cannot find the logic yet notably on regional level. And exactly that fact makes the
“burden” a real burden and difficult to create flexible solutions.
The three suggestions can easily be implemented simultaneously.
Finally, the Interim Evaluation has noted that the following Outcome Indicators can be merged into
single indicators without any further difficulties:
 Indicator 15, 20, 21 could be merged (graduates)
 Indicator 3 and 5 could be merged (female and young entrepreneurs)
 Indicator 13 and 17 could be merged (enrolment rates)
That easy trick would reduce the number of Outcome Indicators by four. The Interim Evaluation has
further considered merging some of the indicators into one single indicator on “employment rates” (see
summary table on Result Indicators below). However, that step necessitates a redefinition of the
selected “Result Indicators”.
81
That goes, strangely, notably for the CGMTs by the way, which might be another point to add to our study of
the management structure. It is a complete puzzle why the central and key stakeholders were caught by surprise
when presented to the Result Indicators of the HRD OP. The Interim Evaluation did not follow up on the point
except from this very light note.
225
IV.2 Summary of the examination of Output Indicators in Part II
IV.2.1 The score system used
SCORE
A
B
C
0
INTERPRETATION
The indicator is understood and acknowledged. The indicator is workable, indicating that smaller changes in
definition might be needed in order to make it measurable and practically
The indicator is understood. The indicator is workable, but changes in definition or ways to generate a measurable
indicator is needed in order to make it work for the evaluator / monitoring expert
The indicator does not work in its present form and needs to be sharpened or revised in order to work from an
evaluator’s point of view
The indicator does not work in its present form either because it is not measurable or because the measures need
to be defined
Count
17
Pct.
18%
31
33%
43
45%
4
4%
IV.2.2 Summary table on Output Indicators
Count /
Ref.
1
2
HRD OP
OIS
SCORE
Number of women
participated in
entrepreneurship
courses for women
Number of women
participated in
entrepreneurship
courses
B
Number of women
participated in child /
elderly care trainings
Number of women
participated in
child/elderly care
training
C
Conclusions / comments





It is extremely difficult to operate with
indicators where the baselines can be
doubted.
Furthermore, we cannot see what is HRD
OP generated and what is basic ALMM
It appears slightly difficult to understand
the indicator and from where it originates
as the priority, the measures, and the
specific objective never address a certain
political focus on driving women into
childcare and elderly care.
It sounds strange that the labour markets
of the 12 NUTS Regions should have
address a specific or rather exclusive
demand for women in childcare and elderly
care; why not men?
We cannot see how the indicator aligns
with the overall horizontal issue of equal
opportunities
Recommendation
Clean the indicator through measures of QA in
data collection. Define its focus (what it appear to
measure) and create a doable baseline
somewhere which appears convincing (the present
baseline is no way convincing)
In general, remove indicators which appear to
contradict the horizontal principles and tend to
distort or create biases onto the labour market
without any reason. It is highly recommended to
assure objective Indicators.
226
Count /
Ref.
3
4
HRD OP
OIS
Number of women
who participated in
employment oriented
courses for women
Number of women
participated in
employment
guaranteed courses
for women
B
Number of women
benefited from
guidance and
counselling services
Number of women
benefited from
guidance and
counselling service
Number of young
people who
participated in
entrepreneurship
courses
Revised: Number of
analyses activities of
the grant scheme
results including the
main lessons learned
Revised: Number of
compendiums
published for
successful granted
projects and results
Number of young
people who
participated in
entrepreneurship
courses
5
6
7
SCORE
Conclusions / comments


The indicator requires solid data similar to
the data provided by İŞKUR
The data has to be cleaned and cannot be
used in its raw form (data from 2006 and
onwards)
B

Same as no. 3
C

Not particular sharp and precise (what
defines “analyses” and what define
“lessons learned”)
Can hardly be measured consistently

Recommendation
Keep the indicator on female participation on
employment oriented training courses provided
that the data are validated, consistent, and based
upon solid QA measures of control.
An additional comment would be to measure both
female and male participants in all employment
oriented training courses in the monitoring reports,
and not just Number of women having participated
in employment oriented training courses for
women, since every study needs a control group
for indicating the effect of the specific programme.
Same as no. 3; i.e.:
Keep the indicator on Number of women benefited
from guidance and counselling services provided
that the data are validated, consistent, and based
upon solid QA measures of control. Again the
additional would be to ensure data on both
genders as well as obtain data from other Turkish
regions in order to ensure control groups.
Redefine the indicator and make it measurable
C

Is number of compendiums really an
essential measure? The indicator
completely lacks measures of quality
Consider the significance and the relevance of
including the indicator which has no measure of
quality connected to it. Include measures of quality
C


Strange divergences in targets
No raw data available which makes the
indicators highly doubtful
In principle the same comments as
indicator 1
There is basically no theoretical or logical objection
towards the indicator insofar consistent data is
available. It is highly recommended to find ways to
agree on the target (as well as the achievements).
An additional note could be that the indicator
clearly matches indicator 1 under specific objective
1 studied above; it makes sense to have indicators
which are fruitful for multiple purposes. Further, it
would again be recommendable to ensure valuable
control groups to see the net effect of the measure

227
Count /
Ref.
8
9
10
11
12
13
HRD OP
OIS
Number of young
people who
participated in
employment
guaranteed courses
Number of young
people who
participated in
internship and
apprenticeship
programs
Number of young
people having
benefited from
guidance and
counselling
Number of
established local
partnerships
Number of young
people who
participated in
internship,
employment
guaranteed courses
and apprenticeship
programs
B
Number of young
people having
benefited from
guidance and
counselling
Number of
permanent local
partnerships to
promote registered
work established and
operating
C
Revised: Number of
SSI, relevant
institutions' staff and
social partners who
participated in
trainings to promote
registered
employment
improved.
A
Number of SSI staff at
central and local level
who participated in
trainings
Number of staff from
social partners who
participated in
trainings on National
Strategy
SCORE
Number of İŞKUR
staff who participated
in PES trainings



B
A









C
14
Conclusions / comments



Recommendation
No raw data
The baseline aggregates the three named
indicators
We cannot be sure that the same
participants are not counted twice
But more so, it is generally not particular
fair and just to separate the counting into
three brackets since they are mutually
related to an obvious extent
Aggregate the indicators into one indicator but
keep the three indicators on an internal operational
level, providing strict guidelines to coding and
quality assurance (QA)
Eliminate the third indicator on guidance when
related to the other two indicators but keep the
indicator as measuring separate actions only
related to guidance and counselling (that would
presumably necessitate a far lower target)
Ensure solid guidelines to separate the two former
actions from each other, and make explicit notes to
what to be reported and measured in case of dropouts
No raw data received
The OIS sharpen the indicator, which is
good
No baseline (except from “0”) which is very
doable and okay
Some mess in targets and data, including
between MIS and OIS
No raw data received
The OIS sharpen the indicator, which is
good
No baseline (except from “0”) which is very
doable and okay
Some mess in targets and data, including
between MIS and OIS
Clean the data. Otherwise, okay indicator insofar it
makes sense for the stakeholders
System-related indicator, which
necessitates relation to impact
Not particular qualitative / sharp
General comment (indicator 12-31): Threat
of too many indicators
Clean the data. Otherwise, okay indicator insofar it
makes sense for the stakeholders
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
228
Count /
Ref.
15
16
17
18
HRD OP
OIS
Number of Monitoring
Mechanisms
Number of monitoring
mechanisms
established for
ensuring the
sustainability of
monitoring of ALMP
through cooperation
with Provincial
Employment and
Vocational Training
Boards.
Number of İŞKUR
provincial offices
transformed into
model offices by
being modernised
and becoming userfriendly to provide
customer-focused
public employment
services
Number of İŞKUR
staff in newly
established model
offices that increased
their
knowledge/abilities
on job search
methods, customer
relations and
managerial skills
Number of İŞKUR
staff that increased
their
knowledge/abilities
on conducting labour
market analyses
SCORE
B
Conclusions / comments
Recommendation

System-related indicator, which
necessitates relation to impact
“Monitoring mechanisms” have to be
defined clearly which necessitate a solid
and consistent guideline allowing the
evaluator to use the indicator
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
“Model offices” and “user-friendly” have to
be defined clearly which necessitate a solid
and consistent guideline allowing the
evaluator to use the indicator
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)


C



C



C



229
Count /
Ref.
19
20
HRD OP
OIS
Number of İŞKUR
and MoLSS staff at
central level
responsible for IT
related issues that
increased their
knowledge/ability on
Software
development training,
Security Training,
Database Training,
and Network Training
Divided into two: 1)
Increase in the
percentage of
qualified software
relating to labour
market information
system of İŞKUR
SCORE
C
Conclusions / comments
Recommendation

System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: Is it really relevant for the HRD
OP??
General comment (indicator 12-31): Threat
of too many indicators
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: Is it really relevant for the HRD
OP??
General comment (indicator 12-31): Threat
of too many indicators
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)


C



21
2) Increase in the
percentage of the
usage of software
quality standard
C



22
Revised: Increase in
the percentage of
qualified Internet
based services of
İŞKUR
C



The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
230
Count /
Ref.
23
24
25
26
HRD OP
OIS
SCORE
Conclusions / comments
Recommendation

System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator is not clear and can hardly be
measured from a qualitative viewpoint
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
Number of İŞKUR
staff at provincial
levels that increased
their
knowledge/abilities
on job and vocational
guidance services
C
Result in the OP,
Output in the OIS:
Number of the
unemployed people
benefited from job
and vocational
guidance services
and increased their
knowledge on job
and career
perspectives
Number of İŞKUR
staff responsible for
designing the Turkish
Occupational
Dictionary that
increased their
knowledge/abilities
on ISCO 2008 to
adopt Turkish
Occupational
Dictionary
Number of İŞKUR
and MoLSS staffs at
provincial levels that
increased their
knowledge/abilities
on monitoring ALMP
C

C





C



The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
The indicator has to be redefined
231
Count /
Ref.
27
HRD OP
OIS
Revised: Increase in
the percentage of
effective of ALMP
measures
determined in the 15
growth centres.
SCORE
C
Conclusions / comments
Recommendation

System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
System-related indicator, which
necessitates relation to impact
Not particular qualitative and sharp
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)


28
29
Number of the
members of
Provincial
Employment and
Vocational Training
Boards and the
experts working in
cooperation with
these Boards who
increased their
knowledge/ability
regarding
employment issues
Number of staff of
MoLSS, İŞKUR and
representativeness of
social actors who
increased their
knowledge on
European
Employment
Strategy,
employment policies
of the Member States
(best practices),
urgent employment
policy measures in
case of global crisis
and the Member
States practises in
designing
employment policies
C



C



The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
232
Count /
Ref.
30
HRD OP
31
32
33
34
Number of
educational staff
participated in
vocational guidance
services training
OIS
SCORE
Number of research
centres established
in the library at the
disposal of MoLSS to
assist the staff
designing
employment policies.
C
Number of models
designed cooperation
and coordination
between MoLSS and
Ministry of Science,
Industry and
Technology is
increased regarding
designing
employment policies.
Number of call
centres (7/24
functioning)
established
Number of İŞKUR
staff who gained the
ability and knowledge
on working in a call
centre and informing
people in an efficient
way
C
Revised: Number of
teachers participated
in trainings about
vocational guidance
services at schools
C
Conclusions / comments
Recommendation

System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
System-related indicator, which
necessitates relation to impact
What does “models” mean?
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
General comment (indicator 12-31): Threat
of too many indicators
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary MoLSS practice)
System-related indicator, which
necessitates relation to impact
General comment (indicator 12-31): Threat
of too many indicators
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What is “efficient”?
General comment (indicator 12-31): Threat
of too many indicators
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)






B


C






The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary İŞKUR practice)
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
233
Count /
Ref.
35
36
37
38
HRD OP
OIS
Number of parents
who are given
trainings
Revised: Number of
parents trained by
programmes aiming
at raising the
awareness of parents
on the importance of
education
B
Revised: Number of
NGO members
participated in
trainings aiming at
raising the
awareness of parents
on the importance of
education
Revised: Number of
protocols signed
between public
institutions, social
partners,
municipalities,
universities and
NGOs aiming at
increasing
cooperated activities
for supporting
education of girls
Revised: Number of
parents families
reached through
campaigns aiming at
raising their
awareness on the
importance of
education for
persuading them to
send their daughters
to school
C
Number of NGO
members participated
in trainings
Number of protocols
between local
administrations,
enterprises, schools,
NGOs and social
partners in order to
increase girls’
enrolment rates
Number of families
individually visited to
persuade to send
their daughters to
school
SCORE
Conclusions / comments
Recommendation


The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)




C




A


No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
Furthermore: It is not clear what role NGO
Members have in relation to the priority
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What is “protocols” and is there no
requirement to “content” and quality?
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)


The indicator needs to be sharpened and
focused towards HRD OP (there is a need to
separate it from ordinary practice)
Make sure that the indicator is relevant for
HRD OP!
None, except for perhaps adds some requirements
for quality
234
Count /
Ref.
39
HRD OP
OIS
Number of pilot
projects implemented
to increase the
quality of VET
education and girls’
schooling and quality
education
SCORE
B
Conclusions / comments
Recommendation


The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
Quality needs to be defined


40
41
42
Revised: Number of
VET programmes
prepared according
to the regional needs
with the cooperation
of all partners
B
Revised: Number of
schools (secondary
education institutions
especially those for
girls) equipped in the
pilot provinces to
contribute to girls’
schooling and quality
education
C
Number of detection
and monitoring
services for students
particularly for girls
under the risk of
dropping out or who
have already
dropped out in
primary and
secondary schools
C











No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What is “quality”?
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
System-related indicator, which
necessitates relation to impact
“All partners” is either too ambitious or
simply non-sense
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What does “equipped” mean? /
What does “quality” mean?
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: There is no indication of what
defines a detector!
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
Define a minimum target of “partners consulted”





The indicator needs to be sharpened and
focused towards HRD OP (there is a need to
separate it from ordinary practice)
Define “equipped” and make a minimum of
requirements to be targeted (sharply) in order
to help the evaluators
Define “quality”
The indicator needs to be sharpened and
focused towards HRD OP (there is a need to
separate it from ordinary practice)
Define “detection and monitoring services”
and make a minimum of requirements to be
targeted (sharply) in order to help the
evaluators
235
Count /
Ref.
43
44
HRD OP
OIS
SCORE
Number of developed
/ revised and
implemented modular
programs in
secondary and postsecondary vocational
schools by region, by
training institution /
school and by domain
Number of
comprehensive and
scheduled awareness
raising events
organised with the
cooperation of private
sector to increase
awareness raising
among the sector by
region and by
economic sector
Number of modular
VET curricula
developed and/or
revised on Judicial
Practices,
Agriculture, Maritime
and Health areas
A
Number of
communication
strategies developed
through awareness
raising activities
B
Conclusions / comments







45
Number of
partnership protocols
between vocational
institutions, social
partners and private
sector in 12 NUTS II
regions
Number of
partnership protocols
between vocational
institutions, social
partners and the
private sector in 12
NUTS II Regions
B




46
Number of secondary
and post-secondary
VET schools with
improved capacity
and programs by
region and by training
institution / school
Revised: Number of
modular VET
curricula developed
and/or revised on
Judicial Practices,
Agriculture, Maritime
and Health areas
A



Recommendation
No raw data received
System-related indicator, which
necessitates relation to impact
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
System-related indicator, which
necessitates relation to impact
The indicator appear slightly weak and
could use some more sharp and precise
definitions: Perhaps there is a link
somewhere between awareness raising
activities and development of
communication strategies (if not directly
being the same); in that case it would
certainly help to know
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
System-related indicator, which
necessitates relation to impact
It would perhaps be nice to indicate some
requirements to “partnership protocols”; it
must be more than just a piece of paper
with some signatures
Otherwise: OK, since it just call for a
quantitative number irrespectively of the
quality
No raw data received
System-related indicator
Same as no. 43
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
236
Count /
Ref.
47
HRD OP
Revised: Number of
teachers participated
in technical training in
the sector for
increased rate of
pedagogical and
professional
qualifications of
administrators and
teachers
Revised: Number of
teachers participated
in trainings for
increasing the quality
of vocational and
career guidance
services in secondary
vocational education
institutions and postsecondary higher
schools improved
48
49
OIS
Revised: Number of
teachers participated
in Teacher Training in
line with the newly
updated curriculum
SCORE
C
Conclusions / comments
Recommendation


No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)

C



C



237
Count /
Ref.
50
HRD OP
OIS
Participants certified
and tested in terms of
vocational knowledge
and skills in 10 VOCTEST by region and
by VOC-TEST Centre
Revised: Number of
QASs in line with the
European Quality
Assurance Reference
Framework for VET
(EQARF) developed
and piloted In the
selected 20 postsecondary higher
schools and 30
vocational
educational
institutions and
revised according to
the results of the
piloting.
Revised: Number of
departments
provided with
equipment support
that are providing
training in the fields
of Judicial Practices,
Agriculture, Maritime
and Health in the
selected 20 pilot
post-secondary
higher vocational
schools and
METARGEM
3000 participants
certified and tested in
terms of vocational
knowledge and skills
in 10 VOC Test
Centres.
51
52
SCORE
Conclusions / comments
Recommendation
A


No raw data received
System-related indicator, which
necessitates relation to impact
The indicator needs to be slightly sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
B


No raw data received
System-related indicator, which
necessitates relation to impact
It is not entirely clear what “equipment
support” means
The indicator needs to be slightly sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
No raw data received
The OIS transformation is not an indicator:
It is a target!
Otherwise OK; it is just a simple number
The indicator needs to be slightly sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine OIS

B



238
Count /
Ref.
53
54
55
HRD OP
OIS
Number of Vocational
Standards developed
by VOC-TEST
Centres and endorsed
by VQA by VOCTEST Centre
Minimum 11
Occupational
Standards endorsed
by VQA
Number of
participants certified
on basic skills
courses by region and
by VOC-TEST Centre
Trainings on basic
skills and further
development of basic
skills for the adults
particularly for
women in formal and
non-formal vocational
institutions in 12
NUTS II Regions
provided.
1800 participants in
employability courses
by these networks
Number of people
who participated in
employability courses
provided by networks
SCORE
C
Conclusions / comments




B



C




56
Number of teachers
who participated in
the technical training
in the sector by region
900 teachers
participated in the
technical training in
the sector
C





No raw data received
The OIS transformation is not an indicator:
It is a target!
“Developed” is not defined; there must be
some minimum requirements
Is it occupational standards or vocational
standards we are measuring? Shall we
agree on “occupational standards” (since
“vocational standards” tend to be nonsense)
No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
“Number of…” has to be added to the
context
No raw data received
The OIS transformation is not an indicator:
It is a target!
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed:
What separate an “employability course”
from any other course
Network is not defined
No raw data received
System-related indicator, which
necessitates relation to impact
The OIS transformation is not an indicator:
It is a target!
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed
Recommendation



The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine OIS
Indicate ways to measure “developed”
The indicator needs to be sharpened and focused
towards HRD OP (there is a need to separate it
from ordinary practice)






The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine OIS
Define “employability course”; there must be
some basic requirements, otherwise forget it
Define “network”
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine OIS
239
Count /
Ref.
57
58
59
HRD OP
OIS
Number of people
who participated to
internships provided
by these networks
(60-65% of
participants are
female, 40-35% are
male.20-25 % of
participants are
graduates of primary
education, 80-75%
are secondary
education)
Number of teachers
trained on new
methods of
monitoring, evaluation
and measurement for
LLL
1200 participants to
internships provided
by these networks
Number of additional
training activities on
basic skills and further
development of basic
skills for the adults,
particularly for women
SCORE
C
Conclusions / comments




Revised: Number of
teachers trained on
new methods for
monitoring,
evaluation and
measuring LLL
activities that are
developed, tested
and implemented.
Revised: Number of
“trainings on basic
skills and further
development of basic
skills for the adults
particularly for
women in formal and
non-formal vocational
institutions in 12
NUTS II Regions
provided.”
0



C



Recommendation
No raw data received
The OIS transformation is not an indicator:
It is a target!
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed:
Network is not defined

No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “new”; “LLL
activities”; “developed”; “tested”; and
“implemented”?
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “trainings” and how
to measure “further development”???






The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine OIS
Define “network”
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine indicator
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Redefine indicator
240
Count /
Ref.
60
61
65
66
HRD OP
OIS
Number of networks
among formal and
non-formal
educationproviders,
business life, NGOs
and social partners
providing courses for
increasing
employability of
people
Number of employees
who participated in
trainings on work
organisation following
training needs
analysis.
Number of focus
groups networks
established
SCORE
C
Conclusions / comments



Revised: Number of
empoyees
partiticipated in
trainings
Revised: Number of
study visits by SME
empoyees who have
who have
participated and
successfully
completed the
trainings to SMEs in
an EU country
Revised: Number of
SMEs visited in
developed and
industrialized
provinces with the
participation of
employers who have
participated and
successfully
completed the
trainings
B
0
C







Recommendation
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “focus groups
network” and how to measure them???

No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “trainings”?

No raw data received
Why invite employees on study visits???
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “trainings” and
“trainings to SMEs” not least; how to
measure “successfully”?

No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “trainings”; how to
measure “successfully”?





The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
The indicator needs to be sharpened in
general in order to avoid an arbitrary and
non-consistent monitoring and evaluation
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
The indicator needs to be sharpened in
general in order to avoid an arbitrary and
non-consistent monitoring and evaluation
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
The indicator needs to be redone completely
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
The indicator needs to be redone completely
241
Count /
Ref.
67
HRD OP
68
69
62
70
Number of employers
who participated in
trainings organized
following a training
needs analysis
OIS
SCORE
Conclusions / comments
Recommendation
Revised: Number of
staff of chambers,
social partners,
Provincial
Employment and
Vocational Training
Board staff and
ÇASGEM staff
increased their
training capacities to
be trainers
Revised: Number of
gap analysis and
trainings needs
analysis at company
level in each 15
growth centres
B


No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “increased” and how
to measure “their training capacities”?

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
B


No raw data received
Criteria needs to be added in order to avoid
excess studies where there is limited need
“gap” needs to be clearly defined

Revised: Number of
vocational school
teachers participated
in on- the- job
trainings
B
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Criteria needs to be added in order to avoid
excess studies where there is limited need
“gap” needs to be clearly defined
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Revised: Number of
employers
participated in the
trainings
B
Revised: Number of
study visits to SMEs
in EU with the
participation of
employers
A









No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “participation” and
“on- the- job trainings”?
No raw data received
It is by no means clear why employers need
to be trained
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “trainings”?
No raw data received
It would have been nice if there were some
assessment involved in “participation”;
otherwise OK




The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)

242
Count /
Ref.
71
72
73
74
HRD OP
OIS
Revised: Number of
workshops by the
participation of
employers from
developed regions at
sectoral level.
Revised: Number of
visits to selected
SMEs in developed
and industrialized
provinces with the
participation of
employers and
employees
Revised: Number of
models of innovative
and sustainable
forms of work
organisation for
SMEs which have
participated in and
successfully
completed the
trainings
Revised: Number of
SME provided by
training and for the
establishment of
support and
coordination centres
within the chambers
and 1 within TOBB in
each of which nonkey experts and
chamber staff will
work
SCORE
Conclusions / comments
Recommendation
A


No raw data received
It would have been nice if there were some
assessment involved in “participation”;
otherwise OK

A


No raw data received
It appear to be captured by the above
indicators indicating no need for additional
measurement

Delete the indicator or make a combined
indicator with nos. 70-72
0


No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “models”;
“innovative”; “sustainable”; “forms of work
organisation for SMEs”; “participation”; and
“successfully”?

The indicator needs to be defined; it cannot
be measured in its current form
0


No raw data received
The indicator is not understood; what does
“Number of SME provided by training and
for the establishment of support and
coordination centres” mean???
What does “1 within TOBB in each of which
non-key experts and chamber staff will
work” mean???
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “training”;
“establishment”; “support”; “coordination”?

The indicator needs to be defined; it cannot
be measured in its current form


243
Count /
Ref.
75
63
76
HRD OP
Number of social
partner staff
participated in training
for trainers
OIS
SCORE
Conclusions / comments
Recommendation
Revised: Number of
awareness raising
meetings for raising
the awareness of
employers and
employeeson the
need for more
training and
consultancy for
increasing
adaptability and
competitiveness
Revised: Number of
chamber staff
accredited as trainers
B


No raw data received
The indicator needs a measurable
dimension of outcome of meetings

The indicator needs to be sharpened
A

No raw data received

Revised: Number of
study visits by
chamber staff and
centre coordinators
to the training and
consultancy centres
esteblished by a
chamber in the EU
A


No raw data received
It would be nice to know why the indicator
only counts “chamber staff and centre
coordinators”
Perhaps there needs to be a focus on
defining the content of the visits

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
The indicator needs to be slightly sharpened

244
Count /
Ref.
77
64
HRD OP
OIS
Revised: Number of
online database
including information
on stakeholders,
resources, the
training needs,
training providers,
announcements
regarding career
opportunities,
decisions of
Provincial
Employment and
Vocational Education
Boards that will serve
for chambers, SMEs,
civil society
organisations and
vocational schools is
established
Number of institutions
(NGOs, trade unions,
public institutions etc.)
benefited from
consultancy services
on the improvement
of the adaptability of
employers and
employees for social
partners, universities,
public and private
VET Institutions,
NGOs, public bodies
and enterprises
SCORE
A
Conclusions / comments
Recommendation


The indicator needs to be slightly sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)

C


No raw data received
Baseline is presumed to be “0” and target
“1”; it is a process indicator! Accordingly,
the word “Number of…” appears silly: Just
write “Establishment of…”
System-related indicator, which
necessitates relation to impact
No raw data received
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “benefited”;
“consultancy services”; “improvement”; and
“adaptability”?

The indicator needs to be defined; it cannot
be measured in its current form without
severe difficulties in ways to make consistent
valued measurements
245
Count /
Ref.
78
HRD OP
OIS
Revised: Number of
web- sites
established for
enhancing the
cooperation and
coordination among
relevant stakeholders
SCORE
A
Conclusions / comments




Revised: Number of
conferences
established for
increasing industryuniversity
cooperation
79
80
Number of people
having participated in
guidance and
counselling services
A
B



No raw data received
Baseline is presumed to be “0” and target
“1”; it is a process indicator! Accordingly,
the word “Number of…” appears silly: Just
write “Establishment of…”
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed to measure “successfully achieved”
No raw data received
The indicator is not really sharp, but we
shall assume that everyone knows what we
are measuring (“number of conferences” on
what?; it is easy to make a conference…)
Same as no. 3
Recommendation

The indicator needs to be slightly sharpened,
otherwise very OK

OK, insofar everyone knows what we
measure

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Same as no. 3; i.e.:
Keep the indicator on Number of people
benefited from guidance and counselling
services provided that the data are validated,
consistent, and based upon solid QA
measures of control.
Find and define control groups.
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Keep the indicator on Number of people
benefited from guidance and counselling
services provided that the data are validated,
consistent, and based upon solid QA
measures of control.
Find and define control groups.


81
Number of people
having benefited from
rehabilitation
programmes
B

We would perhaps like to see some
indication of content of the rehabilitation
programmes




246
Count /
Ref.
82
83
84
85
86
HRD OP
Number of people
having participated in
courses in scope of
ALMPs
OIS
SCORE
C
Conclusions / comments


Number of people
who are reached
through the
awareness raising
activities
C
Number of staff of
social protection
institutions and NGOs
who participated in
trainings on services
related to the
disadvantaged
persons
Number of common
monitoring and
analysis system
established
C
Number of staff of
related institutions
and NGOs in that
area who participated
in trainings on the
formation, usage and
implementation of
central database.
C






B




We would perhaps like to see some
indication of content of the rehabilitation
programmes
ALMP is not just a HRD OP oriented
measure; there has to be a way to see net
impact of the HRD OP
“Reached” is not at all precise!
How to define “awareness raising
activities”?
We would perhaps like to see some
indication of content of the awareness
raising activities
No raw data received
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed, mainly in terms of “participation”
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: “common”; “system”
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: “participated”; “trainings”;
“formation”; “usage”; “implementation”;
“central database”
Recommendation




The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Keep the indicator on Number of people
benefited from guidance and counselling
services provided that the data are validated,
consistent, and based upon solid QA
measures of control.
Find and define control groups.
The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)
Definitions are required in order to perform
monitoring and evaluation

247
Count /
Ref.
87
88
89
HRD OP
OIS
SCORE
Number of
disadvantaged
persons registered to
central database,
monitoring and
analysis systems.
B
Minimum number of
trainings for the staff
of the Operating
Structure related to
improvement of
knowledge and
practice of IPA.
Minimum number of
trainings on the usage
of MIS
A
Conclusions / comments




B



90
91
Minimum number of
trainings for OS staff
to improve their
knowledge on the
IPA-ESF and
differences between
them
Minimum number of
site visits for the
monitoring of the
projects
B




A




System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: Is one person registered just a
count or is there some degree of
“disadvantaged” or other parameters which
has to be taken into account? How do we
separate between “almost normal / perhaps
otherwise strong job-seeker” and
“disadvantaged” (are smokers included to
make an extreme example relevant to some
enterprises)?
System-related indicator, which
necessitates relation to impact
It is by no means clear how the
measurement should be done consistently;
clear definitions and guidelines are severely
needed: What defines “trainings”? Is that
just a brief and friendly advice?
It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
“Number of trainings” is not measurable:
Number of participants or number of
training programmes?
What is meant by “usage”?
It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
“Number of trainings” is not measurable:
Number of participants or number of
training programmes?
What is meant by “knowledge”?
It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
Otherwise OK
Recommendation

The indicator needs to be sharpened and
focused towards the priorities of the HRD OP
(there is a need to separate it from ordinary
practice)

No real comments, except for the point that it
might be difficult to measure relevance for the
HRD OP on a simple measure of counts


No real comments. However:
Define “Number of trainings” to make it
measurable
Define “usage” to make it measurable




No real comments. However:
Define “Number of trainings” to make it
measurable
Define “knowledge” to make it measurable

No real comments
248
Count /
Ref.
92
HRD OP
Minimum Number of
evaluation activities
OIS
SCORE
B
Conclusions / comments



93
94
95
Minimum number of
prepared analyses,
studies, strategy
papers, manuals etc.
Minimum number of
capacity building
activities (trainings,
workshops etc.)
Minimum number of
information events
(seminars, workshops
etc.)
B


B



B




It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
“Number of evaluation activities” is not
measurable
It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
“Number of …” is not measurable
It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
“Number of …” is not measurable
It is a target!!! Delete the word “minimum”!
System-related indicator, which
necessitates relation to impact
“Number of …” is not measurable
Recommendation


No real comments. However:
Define “Number of evaluation activities” to
make it measurable


No real comments. However:
Define “Number of …” and the ways to
measure it to make it measurable


No real comments. However:
Define “Number of …” and the ways to
measure it to make it measurable


No real comments. However:
Define “Number of …” and the ways to
measure it to make it measurable
249
IV.3 Summary of the examination of Result Indicators in Part III
IV.3.1 The score system used
SCORE
A
B
C
0
INTERPRETATION
The indicator is understood and acknowledged. The indicator is workable, indicating that smaller changes in definition might be
needed in order to make it measurable and practically
The indicator is understood. The indicator is workable, but changes in definition or ways to generate a measurable indicator is
needed in order to make it work for the evaluator / monitoring expert
The indicator does not work in its present form and needs to be sharpened or revised in order to work from an evaluator’s point
of view
The indicator does not work in its present form either because it is an Output Indicator, it is not measurable, or because the
measures need to be defined
Count
Pct.
10
36%
4
14%
5
18%
9
32%
IV.3.2 Summary table of comments, conclusions, and the stakeholders’ view on the Result indicators studied
Count
HRD OP
OIS
“Real name”
Type
OK
1
2
Increased rate of
women having
entered into labour
market (having the
qualification and
motivation to seek
a job) after
participating in
courses or
counselling
Increased rate of
women employed
through the
courses and
counselling
Number of women
in labour force
after participating
in courses or
counselling
Output
Indicator
Number of women
employed after
participating in
courses or after
having received
counsel divided by
total number of
women
participating in
courses and
counselling
Output
Indicator
82 %
Survey
Not
good
7%
Comment
Don’t
know
11 %


73 %
10 %
17 %

Score
“Qualification” and “motivation”
can hardly be measured.
It is recommended to redefine the
indicator to “labour force
participation rate” or “employment
rate”
0
It is recommended to redefine the
indicator to “employment rate”
0
250
Count
HRD OP
OIS
“Real name”
Type
Increased rate of
women
entrepreneurs
Number of female
entrepreneurs
Outcome
indicator
73 %
Survey
Not
good
10 %
Increased rate of
young people
employed following
the courses,
internship and
counselling
activities
Increased rate of
young
entrepreneurs
Decrease in the
rate of complaints
(advice slip) on the
unregistered
employment to the
Insurance
Inspection Board
young people
employed
following the
courses, internship
and counselling
activities
Output
Indicator
78 %
5%
17 %
Number of young
entrepreneurs
Outcome
indicator
58 %
9%
33 %

number of
complaints (advice
slip) on the
unregistered
employment to the
Insurance
Inspection Board
Outcome
Indicator
29 %
18 %
53 %

OK
3
4
5
6
7
Increase in the
number of bilateral
projects on
promoting
registered
employment
between the
relevant partners
Revised:
Percentage of
decrease in the
rate of complaints
(advice slip) on
the unregistered
employment to
the Social
Security
Inspection Board
Comment
Don’t
know
17 %




Output
Indicator
53 %
10 %
37 %
A
0

number of bilateral
projects on
promoting
registered
employment
between the
relevant partners
It is recommended to contact
TÜİK / TURKSTAT in order to
provide solid data
Score

It is recommended to contact
TÜİK / TURKSTAT in order to
provide solid data
The Interim Evaluation is not sure
of the purpose of the indicator
doubt on the ability to reflect a
relevant and sustainable process
The indicator do not address
issues of attracting and retaining
employment
The indicator tells nothing about
registered employment
Data based upon administrative
records
Redefine or delete
A
C
0
251
Count
HRD OP
OIS
“Real name”
Type
OK
8
Increase in the rate
of registered
people in 43 city
centres through the
bilateral projects
and local
partnerships
9
10
Employment rates
Outcome
Indicator
Revised: Increase
in the number of
qualified reports,
policy documents
and actions
produced by the
Provincial
Employment and
Vocational
Training
Number of
qualified reports,
policy documents
and actions
produced by the
Provincial
Employment and
Vocational
Training
Output
Indicator
placement rates
Outcome
indicator
Number of
communication
campaigns
organized for
increasing the
visibility of İŞKUR
Number of
registered
unemployed
Output
Indicator
Increase in the
placement rates of
İŞKUR
11
12
13
The rate of
registered
employees in the
12 NUTS II
regions.
Increased
percentage of girls
accessing to
education
particularly
secondary
education as a
result of
programme
intervention.
Number of
communication
campaigns
organized for
increasing the
visibility of İŞKUR
Revised: Increase
in the percentage
of the registration
rate to İŞKUR
Increase in the
access rate of
education,
particularly
secondary
education for girls
enrolment rate of
females with
special focus on
secondary
education levels
57 %
Survey
Not
good
7%
Comment
Don’t
know
36 %
A
0
63 %
9%
28 %

Data based upon administrative
records issues a warning
B
0
Outcome
Indicator
Outcome
Indicator
No comments
Score
74 %
3%
23 %

No comments
A

No comments
A
252
Count
HRD OP
OIS
“Real name”
Type
Dropout rates for
girls at secondary
education and
VET schools
Outcome
Indicator
64 %
Survey
Not
good
6%
number of girls
graduated from
secondary
education and
VET
the share of
teachers, students
and entrepreneurs
satisfied with the
content and quality
of Vocational and
Technical
Education
Outcome
Indicator
63 %
5%
32 %

No comments
A
Outcome
Indicator
57 %
6%
37 %

The indicator is based upon
surveys which is doable, but
leaves no baseline
Satisfaction is relative and difficult
to measure; it needs to be defined
B
enrolment rates in
vocational schools
Outcome
Indicator
“Demand” is non-sense in the
context and not understood
Enrolment rates are okay
B
OK
14
15
16
17
18
Reduction in the
number of girls
dropping out of
secondary
education and VET
Increase in the
number of girls
graduating from
secondary
education and VET
The level of
satisfaction of
teachers, students
and entrepreneurs
who participated in
the activities
regarding the
content and quality
of Vocational and
Technical
Education.
Increasing
demands for 45
pilot vocational
schools.
Teacher training in
line with the newly
updated curriculum.
Decrease in girls'
dropout rates at
secondary
education and
VET schools
Increased
percentage of the
teachers,
students and
entrepreneurs
satisfied with the
content and
quality of
Vocational and
Technical
Education.
Comment

No comments
A

52 %
4%
44 %


number of
teachers trained in
newly developed
curriculum
Output
Indicator
50 %
6%
Score
Don’t
know
30 %
44 %
0
253
Count
HRD OP
OIS
“Real name”
Type
OK
19
20
21
22
Share of primary
education and
secondary
education
graduates among
participants
certified and tested
in terms of
vocational
knowledge and
skills in 10 VOCTEST Centres
Increased
percentage of
participants who
obtained a
certificate in
vocational
knowledge and
skills
Increased
percentage of
participants who
obtained a
certificate in basic
skills
Increased
percentage of
those who found
work after
completing an
employability
course
54 %
Survey
Not
good
5%
Comment
Don’t
know
41 %
Share of primary
education and
secondary
education
graduates among
participants
certified and tested
in terms of
vocational
knowledge and
skills in 10 VOCTEST Centres
number of
participants who
obtained a
certificate in
vocational
knowledge and
skills
Outcome
Indicator with
reservations
Outcome
Indicator
70 %
4%
26 %
number of
participants who
obtained a
certificate in basic
skills
Outcome
Indicator
67 %
4%
29 %
number of
participants from
employability
courses who
obtains a job after
completion relative
to total participants
on the
employability
courses initiated
Outcome
indicator
62 %
8%
30 %

Score
The Interim Evaluation cannot
see and understand the purpose
of the indicator
Data are not available from TÜİK /
TURKSTAT
The indicator should be redefined
C

“participants” has to be clearly
defined
A

“participants” has to be clearly
defined
A


A
254
Count
HRD OP
OIS
“Real name”
Type
?
“induced”
output
indicator
53 %
Survey
Not
good
6%
?
“induced”
output
indicator
46 %
11 %
43 %

Many reservations (see context)
C
Number of training
activities on the
requirements of
adaptability given
by social partners
to their members
“induced”
output
indicator
60 %
8%
32 %

Many reservations (see context)
C
OK
23
24
25
Increased
adaptability of
workers in terms of
having extra skills
or abilities related
to their position
(percentage of
participants
increasing their
salary or being
promoted or having
extra
responsibilities)
Increased
adaptability of
employers in terms
of having extra
skills or abilities in
order to be
competitive enough
(percentage of
participants
increasing the
number of workers,
change of the
branch of activity )
Increased training
activities on the
requirements of
adaptability given
by social partners
to their members
Comment
Score
Don’t
know
41 %

Many reservations (see context)
C
255
Count
HRD OP
OIS
“Real name”
Type
number of
disadvantaged
persons entered
into labour force
(employed or
unemployed)
following the
rehabilitation
programmes,
courses and
counselling
Output
Indicator
63 %
Survey
Not
good
5%
number of
disadvantaged
persons obtaining
a job after
participating in
rehabilitation
programmes,
courses and
counselling
satisfaction on the
services both in
institutional and
target group level
Output
indicator
65 %
5%
30 %
Outcome
indicator
58 %
11 %
31 %
OK
26
27
28
Increased rate of
disadvantaged
persons entered
into labour market
(having the
qualification and
motivation to seek
a job) following the
rehabilitation
programmes,
courses and
counselling.
Increased rate of
disadvantaged
persons employed
following the
courses and
counselling
services
Increased
satisfaction on the
services both in
institutional and
target group level
(by staff of
institutions and by
disadvantaged
persons)
Comment
Score
Don’t
know
32 %
0
0


The indicator is based upon
surveys which is doable, but
leaves no baseline
Satisfaction is relative and difficult
to measure
B
256
Appendix to the study on Indicators: Well-developed indicators
INDICATOR
Unemployed, total
- by gender
- By age groups
- By duration of unemployment, gender and
age
- By education and gender
- By last occupation (ISCO) and gender (or
work experience)
- By criteria on special needs, gender and
age
- By Other criteria, e.g. organised/nonorganized (members of trade unions)
Unemployment rate, total
- by gender
Long-term unemployment rate
- by gender
Unemployment gender gap
UNIT
No.
No.
No.
No.
Unemployment rate gaps for people at a
disadvantage
Ratio
Unemployment rate gap between non EU
and EU nationals
Inflow into long-term unemployment
Ratio
Youth unemployment ratio
Ratio
Employed, total
- by gender
- By age groups
- By occupation (ISCO) and gender
- By economic sector (NACE) and gender
No.
No.
No.
No.
No.
- By other criteria, including self employed,
managers, etc. (organised/non-organized)
No.
COMMENTS (DEFINITIONS, ETC.)
No.
No.
No.
No.
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Employment gender gap
The difference in unemployment rates between
women and men in percentage points
Difference in unemployment rates for disadvantaged
groups (according to national definitions) and the
overall unemployment rate, in percentage points
Unemployment rate gap between non EU and EU
nationals, in percentage points
Share of young/adult unemployed becoming
unemployed in month X, still unemployed in month
X+6/12 without any break of more than one month
(28-31 calendar days).
Total unemployed young people (15-24 years) as a
share of total population in the same age bracket (by
gender)
The difference in employment rates between men
and women in percentage points
Employment rate
- by gender
- By age groups
Employment gender gap in full-time
equivalent
Ratio
Ratio
Ratio
Ratio
Employment gender gap by age group and
educational attainment
Ratio
Gender segregation 1
Ratio
1. Gender segregation in occupations, calculated as
the average national share of employment for
women and men applied to each occupation;
differences are added up to produce a total amount
of gender imbalance presented as a proportion of
total employment (ISCO classification).
Gender segregation 2
Ratio
2. Gender segregation in sectors, calculated as the
average national share of employment for women
and men applied to each sector; differences are
added up to produce a total amount of gender
imbalance presented as a proportion of total
employment (NACE classification).
The difference in employment rates measured in
full-time equivalent between men and women in
percentage points
The difference in employment rates between men
and women in percentage points, by age group (1524, 25-54, 55-64) and by education level (less than
upper secondary, upper secondary and tertiary
education, according to the ISCED classification)
INDICATOR
Employment rate full-time equivalent
UNIT
Ratio
COMMENTS (DEFINITIONS, ETC.)
Total hours worked divided by the average annual
number of hours worked in full-time jobs, calculated
as a proportion of total population in the 15-64 age
bracket.
Employment rate in services
Ratio
Number of employed persons working in the
services sector (in main job) aged 15-64 as
percentage of the population of the same age group.
(by gender)
Labour Force, total
- by gender
- By age groups
Activity rate
No.
No.
No.
Ratio
Labour supply growth
Ratio
Inactive population, total
- by gender
- By age groups
- By main reason for inactivity
No.
No.
No.
No.
Social burden I
Social burden II
Dependent Elderly
Ratio
Ratio
Ratio
Average exit age from the labour force
Ratio
The average age of withdrawal from the labour
market, based on a probability model considering
the relative changes of activity rates from one year
to another at a specific age
Labour market gaps for disadvantaged
groups
No.
Gaps on the labour market (such as employment,
unemployment and inactivity gaps), for
disadvantaged groups (such as disabled people,
ethnic minorities, immigrants, low skilled people,
lone parents, etc.) according to national definitions
Transitions into employment and training
Ratio
Transitions by pay level
Ratio
Transitions of unemployed people into employment
and training from year n to year n+1
Transitions between non-employment and
employment and within employment by pay level
(gross monthly earnings) from year n to year n+1
Transitions by employment status
Ratio
Diversity of contractual and working
arrangements
Ratio
Transitions by type of contract
Ratio
Transitions between non-employment and
employment and within employment by type of
contract from year n to year n+1
Working poor
Ratio
Number of working poor as % of working population,
calculated separately for wage and salary
employees and self-employed. (by gender)
Share of employed and unemployed persons in total
population of working age 15-64
Annual change in labour supply (including employed
and unemployed in working age 15-64). (by gender)
E.g. retired, students, children, disabled or early
retired, ‘housewives’
Number of Inactive relative to no. of Labour Force
Number of Inactive relative to no. of Employed
Dependent elderly men and women over 75 as a
proportion of all men and women over 75.
Breakdown by: living in specialised institutions, help
at home (other than by the family), and looked after
by the family
Transitions between employment, unemployment
and inactivity from year n to year n+1
Total employees in part-time and/or fixed-term
contracts plus total self-employed as % of persons
in employment. Employees in non-standard
employment (parttime and/or fixed-term) as % of
total employees. (Breakdown by part-time, fixedterm, part-time and fixed-term, by reason, by
gender.)
Total self-employed as % of total persons in
employment. (Breakdown by part-time, by gender.)
258
INDICATOR
Poverty (low-wage) trap
UNIT
Ratio
COMMENTS (DEFINITIONS, ETC.)
The marginal effective tax rate on labour income
taking account the combined effect of increased
taxes on labour and in-work benefits withdrawal as
one increases the work effort (increased working
hours or moving to a better job). Calculated as the
ratio of change in personal income tax and
employee contributions plus change (reductions) in
benefits, divided by increases in gross earnings,
using the "discrete" income changes from 34-66% of
APW.
Breakdown by family types: one earner couple with
two children and single parent with two children.
The marginal effective tax rate on labour income
taking account the combined effect of increased
taxes and benefits withdrawal as one takes up a job.
Calculated as the ratio of change in gross income
minus (net in work income minus net out of work
income) divided by change in gross income for a
single person moving from unemployment to a job
with a wage level of 67% of the APW.
Unemployment trap
Ratio
Taxation on low-wage earners
Ratio
Undeclared work
Ratio
Gender pay gap 1
Ratio
Gender pay gap 2
Ratio
1. Gender pay gap by age group and educational
attainment (difference between men’s and women’s
average gross hourly earnings as percentage of
men's average gross hourly earnings for paid
employees at work 15+ hours).
Gender pay gap 3
Ratio
Employment impact of parenthood
Ratio
Gender pay gap, adjusted for sector, occupation and
age
The absolute difference in employment rates without
the presence of any children and with the presence
of a child aged 0-6 (age group 20-50)
Childcare
Ratio
Children cared for (by formal arrangements* other
than by the family) up to 30 hours a usual week / 30
hours or more a usual week as a proportion of all
children of the same age group. Breakdown by:
Children aged under 3 (0-2 years), Children aged
between 3 years and the mandatory school age
(during the day and outside pre-school hours),
Children aged between mandatory school age and
12 years in compulsory primary (or secondary)
education (during the day and outside school hours).
Lack of care for children and other
dependants
Ratio
Share of persons (age group 15-64) who would like
to work but are not searching for a job / who are
working part-time due to lack of suitable care
facilities, in relation to the total population of the age
group. Breakdown by: lack of care services for
children; lack of care services for dependant adults
(i.e. ill, disabled, elderly relatives or friends); lack of
care services for both children and dependant
adults.
Vacancies, total
- By economic sector (NACE) and occupation
(ISCO)
No.
No.
Tax wedge on labour cost: ratio of income tax plus
employee and employer social contributions
including payroll taxes less cash benefits divided by
the labour costs for a single earner earning 67% of
the APW.
Size of undeclared work in national economy (e.g.
as share of GDP or persons employed)
Difference between men’s and women’s average
gross hourly earnings as percentage of men's
average gross hourly earnings (for paid employees
at work 15+ hours). Breakdown by public and
private sectors.
259
INDICATOR
- By types of vacancies
UNIT
No.
COMMENTS (DEFINITIONS, ETC.)
E.g. Permanent and temporary jobs, New job
creation
Share of job vacancies advertised through Member
States' public employment services that are
accessible on a common platform by jobseekers
throughout the EU
Transparency of job vacancies
Ratio
Vacancies per unemployed
Ratio
LMP expenditure
- by activity
- relative to GDP
Activation of unemployed, total
- by gender
- By age groups
- By duration of unemployment, gender and
age
- By education and gender
- By last occupation (ISCO) and gender (or
work experience)
- By criteria on special needs, gender and
age
- By Other criteria, e.g. organised/nonorganized (members of trade unions)
Preventative services
No.
No.
Ratio
No.
No.
No.
No.
New start (a)
Ratio
New start (b)
Ratio
Share of young/adult unemployed becoming
unemployed in month X, still unemployed in month
X+6/12, and not having been offered a new start in
the form of training, retraining, work experience, a
job or other employability measure. (New start = job
or measure LMP categories 1-7) [target value
0%=full compliance] (by gender)
Activation of long-term unemployed
Ratio
Follow-up of participants in active measures
1
Ratio
Follow-up of participants in active measures
2
Ratio
Number of long-term registered unemployed
participants in an active measure (training,
retraining, work experience or other employability
measure) in relation to the sum of the long-term
unemployed participants plus registered long-term
unemployed (yearly averages). Broken down by
types of measures and gender. (LMP categories 27)
1. Rate of inflow of LMP participants into
employment (3 or 6 months after participation in a
measure)
2. Rate of return of LMP participants into
unemployment (3 or 6 months after participation in a
measure)
Enterprises, total
- By Economic Sector
Enterprise births
No.
No.
Ratio
Survival rates of newly born enterprises
Ratio
Trend of the ratio between the total number of the
stock of vacancies compared to the total number of
unemployed (v/u ratio)
Active/passive LMP expenditure as % of GDP
No.
No.
No.
No.
Ratio
Share of young/adult unemployed becoming
unemployed in month X, still unemployed in month
X+6/12, and not having benefited from intensive
counselling and job-search assistance. (LMP
category 1) [target value 0%=full compliance] (by
gender)
Share of young/adult unemployed becoming
unemployed in month X, still unemployed in month
X+6/12, and not having been offered a new start in
the form of training, retraining, work experience, a
job or other employability measure. (New start = a
job or a measure from LMP categories 2-7) [target
value 0%=full compliance] (by gender)
Gross birth rate of new enterprises as a percentage
of total stock of active enterprises
Newly born enterprises of year n that are still active
in year n+3
260
INDICATOR
Employment in newly established enterprises
UNIT
Ratio
COMMENTS (DEFINITIONS, ETC.)
Number of persons employed in newly born
enterprises (in year n) and in surviving enterprises
(set ups in years n-3, n-2 and n-1) in relation to the
number of persons employed in all active
enterprises (in year n)
Work places
No.
Can be indicative when cross-regional commuting
appears
- by economic sector
Trends in accidents at work
No.
Ratio
Trends in occupational diseases
Regional disparities - coefficients of variation:
Employment
Ratio
Ratio
Regional disparities - coefficients of variation:
Unemployment
Ratio
Standard deviation of unemployment divided by the
weighted national average (age group 15-64 years).
(NUTS II)
Regional disparities – Underperforming
regions
Ratio
1. Share of underperforming regions in terms of
employment and unemployment (in relation to all
regions and to the working age population/labour
force) (NUTS II) (by gender).
2. Differential between average
employment/unemployment of the underperforming
regions and the national average in relation to the
national average of employment/unemployment
(NUTS II) (by gender).
Job Satisfaction
Real unit labour costs
Ratio
Ratio
Labour productivity
Ratio
Working time 1
Ratio
Satisfaction with type of work in present job
Growth in total compensation per employee
adjusted for labour productivity and GDP deflator
Total annual output divided by number of occupied
population and hours worked (GDP in PPS per
person employed/per hour worked relative to EU
average)
1. Average weekly number of hours usually worked
per week defined as the sum of hours worked by
full-time employees divided by the number of fulltime employees
Working time 2
Ratio
Overtime work
Ratio
Implicit tax rate on employed labour
Ratio
Population
- By gender
- By age groups
- By economic activity (unemployed,
employed, students, children, retired, etc.)
No.
No.
No.
No.
- By education and gender
- By ethnicity, citizenship or nationality
Day population
Night population
Interregional migration measures
Mortality rates
Birth rates
Fertility rates
No.
No.
No.
No.
No.
Ratio
Ratio
Ratio
The evolution of the incidence rate, defined as the
number of accidents at work per 100 000 persons in
employment
Standard deviation of employment divided by the
weighted national average (age group 15-64 years).
(NUTS II)
2. Trends in average effective annual working time
per employed person.
Number of employees for whom the number of
hours actually worked exceeds the number of hours
usually worked due to overtime as a % of all
employees. (by gender)
Ratio of total taxes on employed labour (personal
income taxes plus employees' and employers' social
security contributions plus payroll taxes) divided by
the total compensation of employees plus payroll
taxes
261
INDICATOR
GDP
UNIT
No.
COMMENTS (DEFINITIONS, ETC.)
GDP growth rates and GDP/c can easily be
calculated if needed. Official statistics is preferred if
available. Regional GDP in absolute numbers can
be used for reference purposes (for instance, in
order to calculate rough productivity measures per
sector).
Annual average of GDP growth
Growth in GDP per capita of employed population
and per hour worked
Relative to employed persons it is a rough indicator
of labour productivity
Relative to employed persons it is a rough indicator
of labour productivity
If wage level by economic sector is available it might
be indicative, but monthly labour costs is
international comparable.
GDP growth
Growth in labour productivity
Ratio
Ratio
Gross Value Added
No.
- by economic sector and region
No.
Monthly labour costs by region
No.
- by economic sector
No.
If wage level by economic sector is available it might
be indicative, but monthly labour costs is
international comparable.
Price indexes by region
Ratio
FDI Foreign Direct Investment by region
Ratio
Production prices (when available); consumer
prices; regional measures (when available)
Relative to GDP. FDI is a highly complex measure
as it is a net measure. In some areas it is yet
another growth indicator.
- by economic sector
Ratio
Relative to GDP. FDI is a highly complex measure
as it is a net measure. In some areas it is yet
another growth indicator.
(Domestic) Gross Investment
Ratio
Relative to GDP. This indicator can be very hard to
obtain on regional level. Net investment might be
available, but this indicator is not as good as gross
investment.
- by economic sector
Ratio
Relative to GDP. This indicator can be very hard to
obtain on regional level. Net investment might be
available, but this indicator is not as good as gross
investment.
Export of Goods and Services
- by economic sector
Import of Goods and Services
- by economic sector
Trade balance
Current Account
Capital Account
Public Expenditure
- by major brackets 1
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
Ratio
- by major brackets 2
Public debt
Pupil/Teacher Ratio (PTR)
Ratio
Ratio
Ratio
Relative to GDP
Relative to total Export
Relative to GDP
Relative to total Import
Relative to GDP
Relative to GDP
Relative to GDP
Relative to GDP
Relative to total public expenditure. By
governmental and local (non-governmental)
Relative to total public expenditure. By sector
Relative to GDP
Average number of pupils per teacher at a specific
level of education, based on headcounts for both
pupils and teachers
- by ISCED levels
Net enrolment ratio in primary schools (MDG
indicator No. 6)
Ratio
Ratio
Gender parity index in primary, secondary
and tertiary education
Ratio
Net primary enrolment ratio is the ratio of the
number of children of official school age (as defined
by the national education system) who are enrolled
in primary school to the total population of children
of official school age
Ratio of female to male gross enrolment ratio. A GPI
of 1 indicates parity between sexes; a GPI between
0 and 1 means a disparity in favour of boys/men; a
GPI greater than 1 indicates a disparity in favour
girls/women.
262
INDICATOR
Gross intake rate to the last grade of primary
school (UIS proxy for Primary completion rate
(MDG goal 2))
UNIT
%
COMMENTS (DEFINITIONS, ETC.)
Total number of new entrants in the last grade of
primary education, regardless of age, expressed as
a percentage of the population of the theoretical
entrance age to the last grade." The number of new
entrants is the total number of children in the last
grade of primary school minus the number of
repeaters
Students first choice of Education
Repetition rates in primary and secondary
schools
%
%
Drop-out rates in primary and secondary
school
%
Percentage of pupils or students who drop out from
a given grade in a given school year. It is the
difference between 100% and the sum of the
promotion and repetition rates.
Early School Leavers
No.
- by types of shools
No.
- by domain
No.
Youth literacy rate (MDG indicator no. 8)
%
Or in fraction of total school population (drop out
rate)
Or in fraction of total school population (drop out
rate)
Or in fraction of total school population (drop out
rate)
Literacy rate of 15–24 year-olds, or the youth
literacy rate, is the percentage of the population 15–
24 years old who can both read and write with
understanding a short simple statement on everyday
life.
Educational attainment of 22 year olds
%
Students
- by domain and gender
Graduates
- by domain and gender
Participation in CVT
Number of agreements between Gvt. (VT
supply) and economic sectors/branches (VT
demand) on the basis of studies of market
perspectives of economic sector/branches
No.
No.
No.
No.
No.
No.
Percentage of training courses updated
further to skills/competencies identified at
enterprise level
%
Percentage of training courses updated further to
skills/competencies identified at enterprise level to
total number of training courses
Number of VT centres built/ rehabilitated/
equipped
No.
Number of VT centres built/ rehabilitated/ equipped
Percentage of trainers trained
%
Ratio VT trainers/VT trainees
Ratio
Enrolment rate in VT as compared to number
of VT applications
%
Percentage of trainers having completed training out
of available trainers in the system
Average number of trainees per trainer, based on
headcounts for both trainees and trainers.
Absorption capacity of training centres as compared
to training applicants
Completion rate
%
Percentage of trainees successfully completing (or
graduating from) VT programs in a given year as
compared to the total number of trainees enrolled
that year
Percentage of VET graduates hired after
training
%
Number of VET graduates hired in a given year,
expressed as a percentage of total VET graduates.
Existence of a demand driven strategy/reform
of VT for employment
No.
A qualitative indicator for measuring of policy
progress, to which the answer is ‘yes’ or ‘no’ within a
timeframe.
Number of repeaters in a given grade in a given
school year, expressed as a percentage of
enrolment in that grade the previous school year.
In percentage of all 22 year olds
Strategic Partnership between government and
professional federations/chambers etc. to improve
market relevance of VT provision, following surveys
undertaken by different sources (internal/external)
on emerging/declining economic sectors.
263
INDICATOR
Public expenditure on VT as a percentage of
Education and Social affairs budget
UNIT
%
COMMENTS (DEFINITIONS, ETC.)
Government spending on Vocational Training as a
percentage of total public expenditure on education
and social affairs.
% of EC contribution within sector reform
financial gap
%
EC spending (funding) for VT as a percentage of
sector reform financial gap (ie the difference
between the financial needs for the implementation
of the reform and the funds available to the
sector/country)
Ratio public sector versus private sector VT
centres
Ratio
Average cost by trainee
Number of studies of market perspectives of
economic sector/branches
Ratio
No.
Number of conventions between Gvt. (VT
supply) and economic sectors/branches (VT
demand)
No.
Strategic Partnership between government and
professional federations/chambers, etc. To improve
VT demand/supply.
Ratio of training courses updated further to
skills/competencies identified at enterprise
level
Ratio
Ratio of training courses updated further to
skills/competencies identified at enterprise level to
total number of training courses
Ratio trainers/trainees
Ratio
Ratio residential/alternate/ apprenticeship
training (initial training)
Ratio
Average number of trainees per trainer, based on
headcounts for both trainees and trainers.
Efficiency of apprenticeship or alternate training
Number of VT centres built/rehabilitated
No.
Number of VT centres built/rehabilitated
Number of courses/CV reoriented towards
'competency based approach' (CBA)
No.
CBA is supposed to reproduce business
environment at VT school /centre level
Number/ratio of trainers trained
No.
Proportion of trainers having completed training
Ratio available training places (supply)/would
be trainees/employees (demand)
Ratio
Number of potential trainees per available training
course.
N./ratio of apprentices hired after training per
skill, per sector and per region
Ratio
Number of VET graduates hired in a given year,
expressed as a percentage of total VET graduates.
Index of stakeholders satisfaction (trainees,
employers, community government) through
surveys
Index
Index associated with the perception of stakeholders
regarding the quality and effectiveness of the VET
program
Increased efficiency / effectiveness of
orientation / information services (rate of
'clients' referred to VT)
%
Percentage of trainees having used the VET
orientation information services
Enrolment rate in VT
%
The number of trainees enrolled in VT programs,
expressed as a percentage of the number of
workers in the corresponding sector.
Literacy rate of managers and employees
%
Number of literate managers and employees,
expressed as a percentage of the total number of
managers and employees
Reduction of % of working children under 15
%
Percentage of children under 15 employed.
Rate of mobility within enterprises
%
Measures the ability of workers/managers to change
easily employment, disaggregated by gender, age
group, profile and sector of economic activity.
Productivity increase within
enterprise/economic branch
Ratio
Output per unit of labor within enterprise/economic
branch
Insertion rate of active population by gender,
age and level of study
%
Labour Insertion rate of active population by gender,
age and level of study
Surveys undertaken by different sources
(internal/external) on emerging/declining economic
sectors.
264
INDICATOR
Distribution of population by level of
education
UNIT
%
COMMENTS (DEFINITIONS, ETC.)
Evaluate changes in the distribution of population
by education level
(Un)employment trends in target
sector/region/group
%
Trends in unemployment rate, ie the percentage of
the total labor force that is unemployed but actively
seeking employment and willing to work
Educational attainment for population (15
years+) by age, gender, ISCED, and Domain
%
Participation rates in TVET as a percentage
of all participants of education/training
%
Percentage of apprentices to total TVET
participants
%
Distribution of apprentices/applied secondary
education by sex and specialization (number
or percentage) to total participants
%
Percentage of vocational education students
to total TVET participants
%
Distribution (number or percentage) of
vocational education students by sex and
type of education.
%
Percentage of students in technical/
technician education to total TVET
participants
%
Distribution (number or percentage) of
participants in technical / technician
education by sex, age and specialisation
%
Gross enrolment rate in TVET by region,
gender, and Domain
Ratio
Net enrolment rate in TVET by region,
gender, and Domain
Ratio
Completion rates in TVET by programme,
and gender
Ratio
Dropout rates in TVET by gender and
programme
Ratio
Dropout rates in CVT by gender and
programme
Ratio
Percentage of graduates in TVET by gender
and programme.
%
Public educational expenditure in TVET to
the total public expenditure
%
Public educational expenditure in CVT to the
total public expenditure
%
Public expenditure per student by type of
education and training
%
Relative proportions of public and private
investment in educational institutions.
%
Funds of TVET by sources and type of
education and training.
%
Funds of CVT by sources and type of
education and training.
%
Cost per contact hour, by programme (all
educations and trainings)
EUR
265
INDICATOR
Graduate cost per programme (all educations
and programmes)
UNIT
EUR
Teaching load per teacher (all educations
and programmes)
Hours
per week
Teachers’ qualifications by gender,
programme and years of experience
?
Expenditure on training and up-qualification
of teaching staff by institution
EUR
COMMENTS (DEFINITIONS, ETC.)
Workplace training: Participation in jobrelated continuing education and training by
labour force status
Workplace training: Participation rates in
continuing education and training by
economic activity and size of enterprise
Ratio
Workplace training: Average duration of
training undertaken by employed adults aged
25–64 in continuing education and training
Days
Participation in adult education / training by
age and sex
%
266
ANNEX B1. List of Meetings held
Regional visits
No
1
1.1
1.2
1.3
1.4
2
2.1
2.2
2.3
2.4
2.5
3
3.1
3.2
4
4.1
4.2
4.3
4.4
4.5
4.6
4.7
5
5.1
5.2
5.3
6
6.1
6.2
6.3
6.4
COUNT
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
13
17
18
19
20
21
22
23
24
INSTITUTION
KASTAMONU
RGMTT
İŞKUR Provincial Directorate
Kastamonu Chamber of Comm. And Industry
Kastamonu Public Training Centre and Evening Art School
VAN
Association for Manufacturer & Enterprise
RGMTT
İŞKUR Provincial Directorate
Van Chamber of Commerce and Industry
Yüzüncüyıl University
SAMSUN
RGMTT
Karadeniz Sanayici ve İşadamları Dernekleri Federasyonu, KASİF
ŞANLIURFA
RGMTT
İŞKUR Provincial Directorate
Şanlıurfa Chamber of Commerce & Industry
MoNE Local Education Authority
Harran University
İŞKUR Deputy Provincial Directorate (repeat)
Karacadağ Development Agency
GAZİANTEP
RGMTT
İŞKUR Provincial Directorate
Gaziantep Commercial Chamber
TRABZON
İŞKUR Provincial Directorate
RDA
Chamber of Commerce
TOKADER, TRABZON
DATE
TIME
13/07/2011
14/07/2011
14/07/2011
14/07/2011
15:00
10:00
11:30
13:30
18/07/2011
19/07/2011
19/07/2011
19/07/2011
20/07/2011
16:00
10:00
14:00
16:00
10:00
19/07/2011
19/07/2011
10:00
16:00
21/07/2011
21/07/2011
22/07/2011
22/07/2011
25/07/2011
26/07/2011
26/07/2011
10:00
14:00
10:00
13:00
14:00
09:30
11:00
27/07/2011
27/07/2011
27/07/2011
10:00
14:00
15:30
09/08/2011
09/08/2011
10/08/2011
10/08/2011
10:00
14:00
10:00
14:00
Meetings with central institutions
List of meetings held on central level
Count
1
2
3
4
5
6
7
8
9
10
No
1
2
3
4
5
6
7
8.1
9
10
INSTITUTION
Ministry of Development
TURKSTAT
TESK
Min. of National Education / CGMT
TISK
SSI / CGMT SSI
TURK-IS
MoLSS: DG Labour and DG OSH
National Qualification Authorities
YOK / CoHE
Operating Structure of HRD OP, MoLSS:
11
11.1
Programme Management Unit
12
11.2
Procurement Unit
13
11.3
Project Management Unit
14
11.4
Financial Management Unit
15
11.5
Quality Assurance and Control Unit
16
11.6
Monitoring and Evaluation Unit
17
11.7
Information, Publicity and TA Unit
18
12
Ministry of Science, Industry and Technology
19
13
EUD
20
14
NIPAC
DATE
21/07/2011
29/07/2011
02/08/2011
03/08/2011
04/08/2011
04/08/2011
09/08/2011
09/08/2011
10/08/2011
10/08/2011
TIME
16:00
10:00
13:30
10:00
13:30
16:00
16:00
10:00
10:00
14:00
11/08/2011
11/08/2011
12/08/2011
11/08/2011
12/08/2011
17/08/2011
12/08/2011
12/08/2011
15/08/2011
15/08/2011
10:00
16:00
15:30
13:30
11:30
11:00
14:00
09:30
14:00
16:00
267
Count
21
22
23
24
25
26
27
28
No
15
16
17
18
19
20.1
20.2
8.2
INSTITUTION
TOBB
HAK-IS
National Authorizing Officer
National Fund
CFCU
İŞKUR: Indicators
İŞKUR: Overall Management and CGMT
MoLSS: DG Internal Audit
DATE
16/08/2011
16/08/2011
17/08/2011
17/08/2011
17/08/2011
18/08/2011
18/08/2011
19/08/2011
TIME
14:00
10:30
13:30
13:30
15:30
10:00
14:00
11:00
Other meetings and similar appearances
List of other meetings held, including management meetings, and other appearances until the date of
submission the draft Interim Evaluation Report
No
1
2
3
4
5
6
7
8
9
10
11
12
Purpose
Kick-Off Meeting
HRD Awareness conference
Weekly Meeting
Weekly Meeting
Weekly Meeting
Monthly Meeting
Information Meeting
SMC Meeting
Weekly Meeting
Weekly Meeting
Weekly Meeting
Monthly Meeting
VENUE
ÇSGB
Hilton, Ankara
ÇSGB
ÇSGB
ÇSGB
ÇSGB
City Hotel, Ankara
Trabzon
ÇSGB
ÇSGB
ÇSGB
ÇSGB
DATE
16/05/2011
18/05/2011
23/05/2011
30/05/2011
13/06/2011
23/06/2011
24/06/2011
07/07/2011
11/07/2011
01/08/2011
15/08/2011
23/08/2011
TIME
10:00
10:00
10:00
10:30
10:30
10:00
13:30
09:30
10:30
10:30
10:30
13:30
List of people met during meetings
The list is reconstructed from lists of participants; not all names and titles have been readable.
CENTRAL LEVEL
Operating Structure, MoLSS
Esat AKTAŞOĞLU, EU Expert Coordinator, MOLSS
Uğurtan TAŞKINER, EU Expert, Programme Management Unit, MoLSS
Ahmet KILINÇ, Prog. Expert, Programme Management Unit, MoLSS
Burcu SAĞLAM, Monitoring & Evaluation Expert, MOLSS
Melahat GÜRAY, EU Expert, Project Management Unit
Ender TAN, Asst. EU Expert, Quality Assurance & Control Unit
İrem YAZAR, Asst. EU Expert, Financial Management Unit
Selin AYTAŞ, Asst. EU Expert, Project Management Unit
Feride Bahar ÖZDEMİR, Financial Manager, MOLSS
Hüseyin Ali Ali TANGÜREK, Financial Management Unit, MoLSS
Melih AKIN, Coordinator, Procurement Unit
Esra DÜZENLİ, Quality Assurance and Control Unit, MoLSS
Salih ENİŞ, Coordinator, Information and Publicity Unit, MoLSS
Şükrü Zafer DOKUZER, Assistant EU Expert Information and Publicity Unit, MoLSS
Hülya TEKİN, Senior Grant Manager, Project Management Unit, MoLSS
EUD
Petek KOVANCU SHEHRIN, Sector Manager
Numan ÖZCAN, Sector Manager
Caner DEMİR, Sector Manager
268
National IPA Coordinator
Nazlı Hezar TANRISEVER, Ministry for EU Affairs, Coordinator
A. Deren Doğan YAVUZ, Ministry for EU Affairs, Director
Üzeyir BAŞER, Ministry for EU Affairs, J. Expert
M. Süreyya SÜNER, Ministry for EU Affairs, Director
National Authorising Officer and National Fund/Treasury
Aykin Berk PAKEL, NAOSD, Expert
Fatih TATLILIOĞLU, NF, Expert
Bayram ÜSTÜNAY, NF, Expert
Gülşen DEMİR AY, NAOSD, Expert
Özkan ÖZKARDEŞ, NF, Expert
Şehnaz ÖZER, NAOSD, DN Chief
CFCU
Aygül ÖZDEMİR, Contract Manager, CFCU
M. Fatih ERKOÇ, Finance Manager, CFCU
Barbaros Murat KÖSE, Coordinator
M.Fatih YILDIZ, Grant Scheme Coordinator
İŞKUR
Tuğba ÇİMŞİR, MHIE, VHKİ
Jülide KUŞCUOĞLU, MHIE, VHKİ
Şerafettin POLAT, MHIE, VHKİ
Halil YILMAZ, MHIE, VHKİ
Nazan KAHRAMAN, İŞKUR, Sef.
Serkan Hasan KOZE, MHIE, VHKİ
Muhittin BOZÖYÜK, MHIE, Mat.
İrfan GÜMÜŞ, Sb Md
Derya Duyar COŞKUN, MHIE, Memur
Secil KABAR, OCU Member, Sef
Kivilcim SARA, OCU Member, Tr. Expert
Ercan AKTEPE, İŞKUR MHIE, Sef
Hakan ÖZ, OCU Member, Tr. Expert
Burhan ARYAN, İŞKUR, PWE OCU
Elif ŞAHİN, İŞKUR, Project Branch Mng
Serkan YÜCEL, İŞKUR, Şb.Md.Yrd.
Participants in Information Meeting
Sibel TUĞ, TISK, Research, Training and External Relations Expert
Burcu Miraç DIRAOR, SPO, Pl. Expert
Gökhan GÜDER, SPO, Head of Department
Irem YAZAR, MOLSS, Assist. EU Expert
Feride Bahar OZDEMIR, MOLSS, Assist. EU Expert
Esat AKTAŞOĞLU, MOLSS, Coordinator of Mon. and Evaluation Unit
Aygül OZDEMİR, CFCU, Contract Manager
Aslı Meryem ŞIVGIN, KOSGEB, Expert
Selin DÜNDAR,
Şahin SERİM, Project Coordinator, HAKİŞ
Yusuf ŞAHİN, Karadeniz Teknik University
269
Ministry of Development
Işıl BOZKURT, Ass. Planning Expert
Serdar POLAT, Planning Expert
Fulya YATMAT, Planning Expert
Sinem ÇAPAR DİRİÖZ, Planning Expert
Ministry of Science, Industry and Technology
E.Meriç Meriçli TAŞAN, RCP-CIC, Head of Prog.Depart
Pınar İRDEM, RCP-CIC, Chief of Peop.Div
Ayşe AYDIN, RCP-CIC, Monitoring Assist.Exp
Sefa MAVİŞ, RCP-CIC, IPA Assist. Exp /MED
Oya ERYİĞİT, RCP-CIC, IPA Assist. Exp /PD
Sayime ÇELİK, RCP-CIC, IPA Assist.Exp /PD
Osman Cihan SELVİ, RCP-CIC, IPA Assist.Exp /PD
Tolga ŞEFLEK, RCP-CIC, IPA Assist.Exp /PD
TÜİK/TURKSTAT
Enver TAŞTI, Head of Department, Social Statistic Department
Güzin ERDOĞAN, Head of Social Sector Group, Social Sector Statistics
Süheyla TULUMBACI, Education Stat. Ass. Expert, Social Sector Statistics
Cengiz ERDOĞAN, Head of Group, Pri. Stat. Group
Murat KARAKAŞ, Head of Group, IYKG
Nevin UYSAL, Team Leader of Vital Stat., Population Demogra.
Ebru GÖK, Expert, Foreign Relation Rep
Vedat METİN, Mathematician, Publication and Data
Dilek ÖZSOY, Head of Group, National Account. Group
Emel URAL, Team Leader, Population and Migration Team
Didem SEZER, Team Leader, Labour Force
Ministry of National Education
İbrahim Nail BURAL, Director of Monitoring & Evaluation Branch
Emrullah ÜNAL, Monitoring & Evaluation Dept.
Elvan KAHYAOĞLU, Director-Increasing Girls’ Educ. Project
Eda ÖZBEK, Expert-Strengthening Secondary Education Project
Murat MIDAS, Expert – Strengthening Special Education
Dr. İsmail DEMİR, Expert - Promoting LLL Project
Namık Kemal YILDIZ, Coordinator – (Girls’ Education Project
Göksel KÖROĞLU, Coordinator of LLL Central Monitoring Team
Kürşat Kutalmış YALÇIN, PCC, Grand Monitoring Coordinator
Social Security Institution, SSI
Varol DUR, Project Coordinator
Ministry of Labour and Social Security
Canan KAMİ, General Directorate of Labour, Assistant Labour Expert
Mehmet Said APUPULAR, DGOHS, Assistant OSH Expert
Selçuk YAŞAR, DGOHS, Assistant OSH Expert
Ceren Seda ERDEM, DG Labour, Labour Expert
Murat ANDAÇ, Auditor, Internal Audit
Mürsel ÇUKUR, Auditor, Internal Audit
Şerif OLGUN ÖZEN, Auditor, Internal Audit
Cengiz ULUTAŞ, Head of the Unit, Internal Audit
270
YÖK /CoHE
Özgül TORTOP
Dr. Ömer AÇIKGÖZ, Advisor
H.Serdar YÜCESU
TESK
Zehra KAYA, Education and Planning Director
Zeynep GAZİOĞLU, EU and Foreign Relations Expert
TİSK
Bülent PİRLER, Secretary General
Sibel TUĞ, Research, Training and External Relation Expert
Tuba Burcu ŞENEL, Research, Training and External Relation Expert
TÜRK-İŞ
Güldane KARSLIOĞLU (Researcher, Assistant Expert)
Namık TAN, Researcher
MYK (National Qualification Authority)
İsmail ÖZDOĞAN
Ahmet GÖZÜKÜÇÜK,
Firuzan SİLAHŞÖR, Head of Occupational Standards Department
HAK-İŞ Confederation, Ankara
Dr. Osman YILDIZ (Assistant to President & Responsible for International Relations)
Jülide SARIEROĞLU
TOBB
Werner GRUBER, Acting Manager, EU Project Development & Monitoring Division
Esin ÖZDEMİR, EU Department Expert
Belgin YILMAZ, Assistant Expert
REGIONAL LEVEL
Kastamonu
Sibel ÖZSAVAŞ, RGMTT Kastamonu
Alpaslan KULUŞ (Data Collection & Control Operator), RGMTT Kastamonu
Muhammed Hayati TABAN (Youth Employment-Project coordinator)
Adnan ARSLAN, Provincial Director, İŞKUR, Kastamonu
Sermin GOLOP, Kastamonu Public Education Centre and Evening Art School
Aydin Faris GÖLOĞLU, Kastamonu Public Education Centre and Evening Art School
Zeliho KUBATOGLU, Kastamonu Public Education Centre and Evening Art School
Zeki GENCOGLU, Kastamonu Public Education Centre and Evening Art School
Cihan CILBIRCIOĞLU, Genel Sekreter Yardımcısı, Deputy Secretary General, Kastamonu Chamber of
Commerce and Industry
Adil Levent BAŞ, KOSGEB, Sinerji Odağı Dış Uzmanı, Kastamonu Chamber of Commerce and Industry
Van
Mahmut GEDİK, Association for Manufacturer and Enterprise, Van
Elvan ÜREY, RGMT, Van
Nurcan ALPASLAN, RGMT, Van
Murat AKKAYUN, RGMT, Van
Abdülkerim ARVAS, İŞKUR, Van
271
Elvan UREY, İŞKUR, Van
Arzu ÇİPLAK, Project Coordinator, Chamber Commerce and Industry, Van
Doç. Dr. Kenan GÜLLÜ, Coordinator, AB-PA office, Yüzüncüyıl University, Van
Samsun
Yaşar BIYIKLI, General Secret., Kasif Office, Samsun
Gülşen GÜLTEKİN, Project Coordinator, Kasif Office, Samsun
Mehmet HUNTURK, RGMTT, Samsun
Gülay ŞENER, RGMTT, Samsun
Hasan KILIÇASLAN, RGMTT, Samsun
Nurcan GÜRSES, RGMTT, Samsun
A.Kemal OZBİŞİRİCİ, RGMTT, Samsun
Soner DOĞANYILDIZ, RGMTT, Samsun
Hicran KARADOĞAN KINIK, RGMTT, Samsun
Burcu F.YAZICI, RGMTT, Samsun
Rukiye DOĞAN, RGMTT, Samsun
Şanlıurfa
Mehmet CENGİZ, RGMT, Şanlıurfa
Halil ŞİLAN, RGMT, Şanlıurfa
Mahmut KAYA, İŞKUR Regional Directorate, Şanlıurfa
Nimet İNCE, Project Coordinator, Chamber of Commerce and Industry, Şanlıurfa
Hanefi ŞAHİN, Educ. Director, Provincial Directory of National Education, Şanlıurfa
Prof. Dr. Abuzer PINAR, Dean, Harran University, Şanlıurfa
Ayşegül ÖZBEK, Specialist, Local office of RDA, Şanlıurfa/Karacadağ Development Agency
Gaziantep
Emirhan Hikmet ASLAN, Regional Monitoring Expert, RGMTT, Gaziantep
İhsan CANPOLAT, RGMTT, Gaziantep
Murat BAĞLIBEL, RGMTT, Gaziantep
İlyas BULDUK, İŞKUR, Gaziantep
Atilla AKARSLAN, İŞKUR, Gaziantep
Şenay YEŞİLTAŞ, Project Assistant, Chamber of Commerce, Gaziantep
Okan ASLAN, Project Coordinator, Chamber of Commerce, Gaziantep
Emel ERÇETİN, Project Coordinator, Chamber of Commerce, Gaziantep
Senay Copur, Project Coordinator, Chamber of Commerce, Gaziantep
Filiz Hosukoglu, Project Consultant, Chamber of Commerce, Gaziantep
Trabzon
Nurettin KARAL, BHIT Coordinator, İŞKUR, Trabzon
Umit ORHAN, General Coordinator, Chamber of Commerce, TRABZON
Zeki Mert BARUT, Chamber of Commerce, TRABZON
Çağlar EMİRZEOĞLU, TOKADER, TRABZON
Onur ADIYAMAN, RDA, TRABZON
272
Annex B2. Terms of Reference
SPECIFIC TERMS OF REFERENCE
Providing Technical Assistance for the First Interim Evaluation of Human Resources
Development Operational Programme
FWC BENEFICIARIES 2009 – LOT 9: Culture, Education, Employment and Social
EuropeAid/127054/C/SER/multi
1. BACKGROUND
2.




DESCRIPTION OF THE ASSIGNMENT
Global objective
Specific objective(s)
Requested services
Required outputs
3. EXPERTS PROFILE
 Number of requested experts per category and number of man-days per expert
 Profile required
4.




LOCATION AND DURATION
Starting period
Foreseen finishing period or duration
Planning including the period for notification for placement of the staff as per art 16.4 a)
Location(s) of assignment
5.




REPORTING
Content
Language
Submission/comments timing
Number of report(s) copies
6.ADMINISTRATIVE INFORMATION
 Interviews if necessary indicating for which experts/position
 When in the interest of the project, possible limits to subcontracting
 Language of the specific contract
 Request for a succinct methodology when needed
 Management team member presence required or not for briefing and/or debriefing
 Other authorized items to foresee under ‘Reimbursable’
 For riders only : operational conditionality for intermediary payment if foreseen as per article 7.2 b) of
the Special Conditions
 Others
ANNEX I: Tax and Customs Arrangements
ANNEX II: Evaluation Grid
273
1. BACKGROUND
With the aim of benefiting from financial assistance provided under the IPA, Turkish Ministry of Labour
and Social Security (MoLSS), designated as the OS (Operating Structure) of Component IV which is
Human Resources Development, have prepared the Operational Programme (HRD OP) covering the
period of 2007-2009 with the involvement of relevant stakeholders. HRD OP was designed to “address
the main challenges Turkey faces in the fields of employment, education and social inclusion” with a
geographical concentration approach, especially on the 12 NUTS II Regions 82 of Turkey.
There are five priority axes defined under the HRD OP:
 The first priority axis aims to attract and retain more people in employment through the
measures of enhancing the employment of women and young people, promoting registered
employment and improving the quality of the public employment services.
 The second priority axis seeks to enhance investment in human capital through education,
especially with the measures of increasing enrolment of girls and the quality of Vocational and
Technical Education (VET).
 The third priority axis was designed for increasing adaptability of both employees and
employers through promoting life-long learning and investing more in human capital.
 The fourth axis aims to the integration of the disadvantaged persons to the labour market
through increasing their employability and contributing to the improvement of the institutions
functioning in the area of labour market and social protection.
 The fifth priority axis of the HRD OP is the technical assistance. It tries to achieve this objective
through the measures to support the HRD OS for preparatory, management, monitoring,
administrative support, information, evaluation and control activities; development of
absorption capacity of final beneficiaries through information and publicity activities.
This assignment will be carried out under the fifth priority axis of the HRD OP which aims to support
the MoLSS as the OS of the HRD OP. Through this assignment, MoLSS will be supported in its duties
related to the interim evaluation of the HRD OP.
2. DESCRIPTION OF THE ASSIGNMENT
2.1. Global Objective
To make an independent evaluation of the overall relevance, effectiveness and efficiency of the
implementation of the HRD OP with a view to provide all the stakeholders with reliable data, analysis
and feedback that will help the upcoming OP revision exercise.
2.2. Specific Objectives
The specific objectives of the assignment are:
- to assess the relevance (in particular with the strategic documents such as the Strategic Coherence
Framework83, Multi-Annual Indicative Planning Document84, employment guidelines, strategic
community guidelines and the Lisbon Strategy), efficiency, effectiveness of the programme.
- to assess the complementarity between the HRD OP and Regional Competitiveness Operational
Programme (RC OP),
- to provide data on the output and result indicators agreed upon in the HRD OP that cannot be
obtained through monitoring system,
- to assess the achievement of the horizontal principles,
- to identify best practices, factors for success and failure and provide recommendations to improve
the remaining programming and implementation of the HRD OP.
2.3. Requested Services
After the signing of the framework contract, following procedures will be pursued:
82
The regions whose GDP per capita is less than 75% of the average GDP per capita of Turkey.
Please seehttp://www.ikg.gov.tr/document.html
84Pleaseseehttp://www.ikg.gov.tr/document.html
83
274
1-Inception phase
The inception phase starts with a kick off meeting to be held at the premises of the Beneficiary. The
aim of the “kick-off” meeting should be, amongst other:
- introduction of the parties and any information or other assistance to be provided by the MoLSS/
Contracting Authority/ the participants to the Consultant,
- discuss any specific issues that require further attention.
After the kick of meeting, within 10 calendar days, the Consultant shall prepare an inception report mainly by
revising the “methodology” that was presented within the proposal on the basis of the feedbacks received
from the MoLSS, Evaluation Sub-Committeemembers of Sectoral Monitoring Committee85, European Union
Delegation to Turkey (EUD), European Union Secretariat General (EUSG) and Central Finance and
Contracts Unit (CFCU).
The inception report shall also include an executive summary and detailed work plan of management
of the assignment including the plan about the visits to the determined Regional Grant Monitoring and
Technical Assistance Teams86 (RGMTTs), time schedule with clear milestones for the implementation,
specific activities to be carried out, identification of risks that might occur and recommendations to
overcome them.
At the end of the 1st month of the assignment, the inception report is expected to be approved.
2- Information Meeting
The assignment will contribute to gaining experience and improving capacity on programme
evaluation. To that aim, a half a day meeting on evaluation of HRD OP for the Internal Coordination
Team within the Beneficiary with the members of the SMC Evaluation Sub-Committee will be
organized by the Consultant after the completion of inception phase.
3-Analysis phase
HRD OP will be evaluated through the evaluation questions listed below and the evaluation method
adopted in the inception phase:
Evaluation Question 1:
What is the overall relevance, efficiency and effectiveness of the HRD OP at the level of priority and
measures in particular as regards;



The appropriateness of the management structures formed at the levels of Operation
Structure and Operation Beneficiaries?
The use and financial allocation of financial assistance?
The quality of projects designed to achieve the objectives, targets and indicators determined
in the HRD OP?
Evaluation Question 2:
What is the complementarity between the HRD OP and RC OP in particular as regards achieving the
objectives, targets and indicators determined in the HRD OP and RC OP?
85
The Sectoral Monitoring Committee, established in accordance with Article 59 of the IPA IR, includes the relevant
institutions functioning within the IPA structure such as Operating Structure for HRD OP, National IPA Coordinator
(NIPAC), National Authorizing Officer (NAO), Strategic Coordinator for Components III and IV (State Planning
Organisation) and EUD as well as the representatives from the social partners and civil society. The Committee meets at
least twice a year and reviews at each meeting the progress made towards achieving the specific targets of the OP.
Evaluation Sub-Committee has been established under the SMC. It shall support the independence of the evaluation
function, review and check the quality of works of external team (using an agreed quality check list) including inception
and draft evaluation reports and issue comments. Evaluation Sub-Committee is composed of representatives selected
among the SMC members.
86With the aim of providing technical assistance to potential and final Grant Beneficiaries and carrying out monitoring of
grant projects at local level, Regional Grant Monitoring and Technical Assistance Teams (RGMTTs) were established
under the responsibility of the MoLSS in 12 provinces within the eligible NUTS-II regions (namely Trabzon, Kars,
Samsun, Erzurum, Gaziantep, Elazığ, Batman, Kastamonu, Şanlıurfa, Sivas, Kahramanmaraş, Van). The RGMTTs are
consisting of at least 9 experts from the provincial directorates of Turkish Employment Agency (ISKUR), Ministry of
National Education (MoNE) and Social Security Institution (SSI).
275
Evaluation Question 3:
What are the output and result indicators agreed upon in the HRD OP that cannot be obtained through
monitoring system?
Evaluation Question 4:
What are the results of the programme in achieving the horizontal principles during the design and
implementation of Operations?
Evaluation Question 5:
What are the best practices, factors of success and failure regarding the planning and implementation
process of the HRD OP? What recommendations can be proposed to the actors involved in the
programming, determining financial allocations, monitoring and evaluation?
These questions will be directed to the target institutions mentioned below:












Operating Structure
Operation Beneficiaries (i.e; Ministry of National Education [MoNE], Turkish Employment
Agency [ЭЮKUR], Social Security Institution [SSI] and The Union of Chambers and
Commodity Exchanges of Turkey [TOBB])
Evaluation Sub-Committee members of Sectoral Monitoring Committee
Ministry of Industry and Trade
Central Grant Monitoring Teams (CGMTs87)
Regional Grant Monitoring and Technical Assistance Teams (RGMTTs)
CFCU
National Fund (Undersecretary of Treasury)
National Authorizing Officer (Undersecretary of Treasury)
Strategic Coordinator for Components III and IV (State Planning Organisation)
National IPA Coordinator (EU Secretariat General)
European Union Delegation to Turkey
The responses given to these evaluation questions by the aforementioned institutions will be the main
source of data for the “Interim Evaluation Report”.
Tasks to be accomplished with respect to evaluation question #1
a. Analysis of Previous Evaluation’s Results

Review the results of the ex-ante evaluation carried out by external independent team under the
responsibility of the Operating Structure within the framework of the EU-supported Technical
Assistance Project “Support to the SPO to Build Capacity at Central, Regional and Local Level to
Implement Economic & Social Cohesion Measures" in September 2007.
b. Analysis of the Relevance of the HRD OP

Analyze and review the major developments since 2007 at national and EU level that influence
the HRD OP,

Analyze the HRD OP’s coherence with the objectives of pre-accession assistance, strategic
documents such as the SCF, Lisbon Strategy, MIPD, employment guidelines, strategic
community guidelines,

Make a SWOT analysis of the beneficiaries and stakeholders,

Analyze the accuracy, clarity, quality, usefulness and internal consistency of the overall
objectives, purposes, targets, indicators and eligible activities at the priority and measure level of
the HRD OP.
87
Each Operation Beneficiary having a Grant Scheme establishes a Central Grant Monitoring Team (CGMT)
composed of minimum 5 experts under the coordination of the OS. The CGMT is co-chaired by an expert from the
Operation Beneficiary and an Operation Coordinator from the OS. The CGMT is responsible for monitoring and
controlling the technical implementation of the projects under the grant schemes.
276
c.
Analysis of the HRD OP’s Effectiveness and Efficiency

Review programme outputs and results in relation with the expenditures incurred at priority and
measure levels; in the light of financial and physical progress,

Collect data on indicators not provided through the regular monitoring system,

Make an overall examination of the HRD OP’s programming, monitoring and evaluation
structures and procedures (in particular of the content, timeliness and compliance) to provide an
assessment of the public administration systems’ efficiency and effectiveness in terms of
constructing the necessary mechanisms for attaining the objectives under HRD OP,

Assess the concrete progress of the HRD OP on the basis of the indicators,

Examine the efficiency of Operation Selection Criteria,

Analyze the role of national bodies, the relations/co-ordination with local authorities, institutions,
beneficiaries and other stakeholders, the commitment of all the stakeholders,

Assessment of local absorption and implementation capacities,

Analyze the financial management of the budget,

Analyze the accuracy of the budget allocations under measures.
Tasks to be accomplished with respect to evaluation question #2
The Consultant should analyze the complementarity between HRD OP and the RC OP and prepare a
thematic report annexed to the “Interim Evaluation Report” by analyzing the extent to which,

RC OP’s objectives have been achieved through programming and implementation of the
HRD OP,

RC OP has influenced the management structures of the OS and Operation Beneficiaries of
HRD OP.
Tasks to be accomplished with respect to evaluation question #3

The Consultant will analyze and provide data on output and result indicators agreed upon in
the HRD OP that cannot be obtained through monitoring system.
Tasks to be accomplished with respect to evaluation question #4
A thematic report annexed to the “Interim Evaluation Report” in order to evaluate the progress made
towards meeting the horizontal principles during the design and implementation of the operations
under HRD OP will be prepared. These horizontal issues are:
 equal opportunities for men and women;
 sustainable development & environmental protection;
 participation of civil society;
 geographic, sectoral and thematic concentration;
 concerns of disadvantaged persons;
 good governance
Tasks to be accomplished with respect to evaluation question #5
The Consultant will analyze and determine the best practices and the factors of success and failure
regarding the planning and implementation process of the HRD OP. Moreover, the Consultant will
propose recommendations to the actors involved in the programming, determining financial
allocations, monitoring and evaluation.
With the completion of the analysis, conclusions should be drawn by the Consultant based on robust
findings of the interim evaluation. It is expected from Consultant to provide recommendations through
follow-up tables for the next programming period of the HRD OP; identify and formulate later phases
on the basis of lessons learned during the evaluation. Recommendations should indicate deadlines,
be realistic and pragmatic. If options for future action are available, these should be identified and
ranked taking into consideration their costs. These conclusions and recommendations will be an
integral part of the Interim Evaluation Report.
4- Preparation of Draft Version of the “Interim Evaluation Report” and its Annexes (Thematic
Reports, Conclusions and Recommendations)
277
The Consultant shall produce the draft version of the “Interim Evaluation Report” and its annexes, on
the basis of the findings of the analysis, three months after the commencement of the assignment.
After Quality Assurance and Control Unit (QACU) within the OS checks the draft report and its
annexes, they will be issued in soft copy for comments of the target institutions listed above and DG
Employment, Social Affairs and Equal Opportunities. The comments on the report and its annexes will
be issued in 10 calendar days.
5- Briefing Meeting
A half-day briefing meeting will be organized by the Consultant within 3 days after the completion of
draft version at the premises of the Beneficiary. The aim of the briefing meeting is to provide
information and receive feedbacks/comments about the draft version of the “Interim Evaluation
Report” from the senior representatives of the target institutions. Findings and recommendations about
the implementation and planning of HRD OP will be particularly stressed in the briefing meeting.
According to the comments received from the target institutions, DG Employment, Social Affairs and
Equal Opportunities and the senior representatives attended to briefing meeting, the Consultant will
make necessary revisions within 5 working days from the receipt of comments.
6- Submission of the Final Version of the “Interim Evaluation Report” and its Annexes
(Thematic Reports, Conclusions and Recommendations)
At the middle of the 4th month, the Consultant will send the revised report to the OS together with a
table indicating how the comments have been handled. QACU of the OS will review the revised
version of the “Interim Evaluation Report” and the soft copy of the report will be submitted to the target
institutions listed above and DG Employment, Social Affairs and Equal Opportunities for feedbacks.
They will provide feedbacks within 10 calendar days. According to the comments received, the
Consultant will make necessary revisions within 5 working days and finalize the preparation of the
report.
Finally, the final version of the report will be discussed in the Sectoral Monitoring Committee meeting,
which is foreseen to be held in June 2011, for approval. Then, the approved “Interim Evaluation
Report” would be sent to the EC.
Totally 10 copies of the final version of the Interim Evaluation Report, with all its annexes will be
published and distributed to CFCU (1 copy), EUD (1 copy), EUSG (1 copy) andMoLSS (7 copies) in
hard copy. However, the rest of the target institutions will be provided with the final version of the
Interim Evaluation Report via e-mail.
After the approval of the “Interim Evaluation Report”, the booklets which include an executive
summary of the “Interim Evaluation Report” of HRD OP will be submitted to the related parties. After
approval of the draft copies, the 50 copies of the booklets will be published and submitted to the
MoLSS to be disseminated to the relevant institutions.
Month
February 2011
Task
Commencement of the assignment
Preparation and approval of the Inception Report
Organising information meeting
March – April 2011
Analysis phase
May 2011
Submission of the draft version of the “Interim Evaluation Report”
Receipt of comments/feedbacks
Organising briefing meeting
Submission of the revised version of the “Interim Evaluation Report”
Receipt of comments and finalizing the preparation of the report
Table 1: Indicative time-table for the assignment
278
With respect to the monitoring structure, proposed methodology will be as follows:
The OS of the HRD OP is responsible from the tasks related to the monitoring and implementation of
the assignment.
Coordination Meetings: Weekly coordination meetings will be held with the participation of Consultant
and Internal Coordination Teamto assess the progress of the assignment and propose corrective
measures. The weekly coordination meetings, which will be held at the OS’s premises in Ankara, will
be chaired by the Beneficiary.
Monthly Coordination Meetings: Monthly meetings will be held with the participation of Consultant,
Internal Coordination Team, Internal Audit Department, Evaluation Sub-Committee Members of SMC,
the CFCU and the EUD in order to advice, guide and control the Consultant. Monthly meetings will be
held at the premises of the Beneficiary.
Consultant will prepare the agenda prior to the meeting. The minutes of weekly and monthly meetings
and submit to the parties involved in 3 working days for feedbacks and comments in order to ensure
up-to date monitoring of the tasks.
2.4. Required Outputs
Types of
indicators
Target (2011)
Source of verification
Increased capacity of MoLSS and EU
Coordination Department in tasks related
to programming, implementing and
evaluation
90% of
recommendations
adopted and
implemented
according to
deadlines set in
the
follow-up tables
Sectoral Monitoring Report, Sectoral
Annual Report, MoLSS Programme
Report, revised HRD OP, monitoring
sheets, final beneficiary reports
Result
Increased awareness of the internal
coordination team within the Beneficiary
and members of the SMC evaluation subcommittee with respect to evaluation of
the HRD OP.
Sectoral Monitoring Report, Sectoral
Annual Report, MoLSS Programme
Report, revised HRD OP, monitoring
sheets, questionnaires
Output
An inception report prepared within 10
calendar days following the kick of
meeting
Approximately 20
people attended
to the information
meeting give
positive
answers to the
questionnaires on
the quality and
usefulness of
meeting delivered
by the
Consultant.
1
Output
An information meeting with the Internal
1
Coordination Team within the Beneficiary
and members of the SMC Evaluation SubCommittee
A briefing meeting with the senior
1 meeting with
representatives of target institutions
the participation
of at least 1
senior
representative
from each of the
structures within
the target
institutions
“Interim Evaluation Report”
1
Number of hard copies of the “Interim
10
Evaluation Report” with its annexes
Number of booklets of executive summary 50
of the “Interim Evaluation Report”
Result
Output
Output
Output
Output
Indicator
Inception Report
Minutes of the meeting, participant
list
Minutes of the briefing meeting,
participants list
Final “Interim Evaluation Report”
Copies
Booklets
279
Output
Number of visits to target institutions for
interviews and data collection
Output
Thematic report annexed to the “Interim
Evaluation Report” assessing the
complementarity between the
implementation of HRD OP and RC OP
Thematic report annexed to the “Interim
Evaluation Report” on meeting the
horizontal issues during the preparation
and implementation of HRD OP
Output
Min. 17
(min. 5 visits to
RGMTTs,
min. 12 visits to
the structures
within the target
institutions)
1
Sectoral Annual Report, Sectoral
Monitoring Committee Report,
Reports received from the
Consultant
1
Thematic Report, “Interim
Evaluation Report”
Thematic Report, “Interim
Evaluation Report”
3. EXPERTS PROFILE
3.1. Number of requested experts per category and number of man-days per expert
The Consultant shall make a team of one senior expert and two junior experts available for the
successful completion of this assignment. 195 working days including the international travel days
will be totally allocated to the experts for the implementation of the assignment. The Consultant shall
ensure that these experts work in cooperation with each other.
The experts and the indicative number of working days of the experts are given in the table below:
No
1
2
3
Experts
Team Leader - Expert on Employment
Expert on Education
Expert on Tendering, Financial Management and
Contract/Project Management
Category
Senior
Junior
Junior
TOTAL
Number of Working Days
65
65
65
195
3.2. Profile required (education, experience, references and category as appropriate)
The experts to be proposed shall have the necessary professional qualifications and experience
required to successfully deal with the issues and to cover all the activities indicated in this Terms of
Reference. If the team proves unable to meet the level of quality required for drafting the reports, the
consulting team shall provide, at no additional cost to the Beneficiary, an immediate technical support
to the team to meet the required standards.
The qualifications, skills and experience listed below are the minimum required skills except the ones
specified as assets.
Expert 1: Team Leader - Expert on Employment
Qualifications and skills:




Bachelor’s degree in economics or social sciences or administrative sciences, or any relevant
field of study linked with human resources development. In the absence of a university degree
the expert will have an equivalent professional experience of minimum 14 years in the relevant
fields.
Graduate/post graduate degree in employment and/or social policy field will be an asset.
Good command of both written and spoken English
Full computer literacy
General professional experience:

At least 10 years of professional working experience in the sector(s) related to the lot.
280
Specific professional experience:






At least 5 years of experience in employment and social policy areas,
Experience in setting out indicators and making statistical analysis, preferably through an
experience in the preparation of an HRD OP,
Experience in an EU-funded project as a team leader/project manager,
Experience in evaluation and/or designation and/or implementation of EC Structural and
Cohesion funds and/or pre-accession assistance programmes in the field of HRD
conducted in EU candidate countries or member states.
Experience in the following area(s) will be an asset:
o Drafting/updating Operational Programmes,
o Preparation of sectoral annual and final reports,
o Checking the progress/monitoring reports of the projects (major projects),
o Programme evaluation.
Specialization on the employment of women, youth and/or disadvantaged people, public
employment services and/or unregistered employment will be an asset.
Expert 2: Education Expert
Qualifications and skills:



University degree in education or social sciences or any other related field. In the absence of
a university degree the expert will have an equivalent professional experience of minimum 7
years in the relevant fields.
Good command of both written and spoken English
Full computer and internet literacy
General professional experience:

At least 3 years of professional experience in the sector(s) related to the lot.
Specific professional experience:
 At least 1 year of experience in evaluation and/or designation and/or implementation of
education programmes related with employment and human resources development,
 Experience in EC Structural and Cohesion funds and/or pre-accession assistance
programmes in EU candidate countries or member states,
 Experience in making statistical analysis,
 Experience in EU projects will be an asset.
 Specialization on vocational training, lifelong learning and/or adaptability of workers/employers
will be an asset.
Expert 3: Tendering, Financial Management and Contract/Project Management Expert
Qualifications and skills:
 University degree in finance, economics or any other related field. In the absence of a
university degree the expert will have an equivalent professional experience of minimum 7
years in the relevant fields.
 Good command of both written and spoken English
 Full computer literacy
General professional experience:
 At least 3 years of professional experience in the sector(s) related to the lot.
Specific professional experience:
 At least 1 years of experience in the field of evaluating and/or implementing financial planning,
financial management and/or budget planning of international programmes,
 Experience in making statistical analysis,
 Experience in tendering procedures executed by the managing authorities / implementing or
intermediate bodies under Structural and Cohesion Funds, especially the European Social
Fund; or executed by the CFCUs in EU candidate countries or member states,
 Experience in the field of human resources development programmes will be an asset.
281
4. LOCATION AND DURATION
4.1. Starting period
The indicative starting period for this assignment is February 2011.
4.2. Foreseen finishing period or duration
The foreseen duration of the assignment is 150 calendar days.
4.3. Planningincluding the period for notification for placement of the staff as per art 16.4 a)
The Consultant, at the beginning of the assignment, will have an initial briefing meeting (kick-off
meeting) at the premises of MoLSS in Ankara with the representatives of the MoLSS, CFCU, EUD,
EUSG and other related institutions. Necessary documents such as HRD OP, RC OP, results of the
ex-ante evaluation will be submitted to the Consultant in this meeting. The kick-off date will be
determined by an administrative order after the signature of the contract.
Work-plan, time-management and organization of the team members (including designation of a Team
Leader and allocation of the duties among experts) for the efficiency and success of the assignment
are the responsibility and duty of the Consultant.
4.4. Location(s) of Assignment
The country of the assignment is Turkey and working place is in Ankara at the premises of the Ministry
of Labour and Social Security. However, min. 5 visits to RGMTTs for interviews are also foreseen. The
RGMTTs to be visited will be listed in the inception report. Except the visits to target institutions for
interviews and data collection, the Consultant will perform the tasks at the premises of the Ministry of
Labour and Social Security/EU Coordination and IPA Management Department.
Address:
ЭnцnьBulvarэ No: 42 C Blok, Kat: 4 PK 06100 Emek/ANKARA
5. REPORTING
5.1. Content
The Consultant should submit the inception report in accordance with the provisions explained in
detail under the “Requested Services”.
The Consultant should also submit a final report that reflects the outcome of all tasks performed. The
final report should include the synopsis of the assignment, observations related to implementation. All
relevant technical papers covering the “Interim Evaluation Report” and the findings of the interviews to
be made with the target institutions including the RGMTTs, reports, and papers prepared from the
start to the end of the assignment will be attached to the final report.
The Consultant should submit these reports to the related parties: the MoLSS, Evaluation SubCommittee Members of SMC, EUD, EUSG and CFCU.
5.2. Language
All reports shall be prepared and submitted in English.
5.3. Submission/comments timing
After the kick-off meeting the Consultant will prepare an inception report within 10 calendar days and
submit it to the related parties (MoLSS, Evaluation Sub-Committee Members of SMC, EUD, EUSG
and CFCU).
The Consultant shall submit the final report not later than 7 calendar days after completion of the
assignment.
282
The MoLSS, Evaluation Sub-Committee Members of SMC, EUD, EUSG and CFCU have 10 calendar
days to provide comments on the submitted reports. Necessary revisions will be made by the
Consultant within 5 working days from the receipt of the comments. In the absence of any comments
from the related parties within 10 calendar days after the submission of the reports, they will be
considered to be tacitly approved.
5.4. Number of report(s) copies
The Consultant shall submit the reports to the related parties for comments in electronic version
readable by a Microsoft Office application. The final versions of the reports will be submitted to the
related parties both in hard copy (one copy to each party,) and in electronic version readable by a
Microsoft Office application.
6. ADMINISTRATIVE INFORMATION
6.1. Interviews if necessary indicating for which experts/position
No interview is foreseen.
6.2. When in the interest of the project, possible limits to subcontracting
No subcontracting is foreseen.
6.3.Language of the specific contract
Language of the specific contract is English.
6.4. Request for a succinct methodology when needed
The tenderers shall propose a methodological approach for this evaluation taking into consideration
the objectives, evaluation questions and quality control requirements. Hence, the proposal should
include an implementation plan and allocation of the duties among experts as well as an “evaluation
methodology” for the programme evaluation such as interviews, focus groups, case study,
counterfactuals to provide robust findings. This list is not exhaustive, any other methods and the
appropriate number of analysis can be proposed.
6.5. Management team member presence required or not for briefing and/or debriefing
Management team member presence is not required for briefing and/or debriefing.
6.6. Other authorized items to foresee under ‘Reimbursable’
The budget for this assignment should include (1) fees, and (2) reimbursable expenses. The
reimbursable costs shall include i) per diems, ii) international travel costs of the experts (6 travels), iii)
inter-city travel costs (min. 10 travels), (iv) translation costs (350 pages), (v) consecutive interpretation
costs for interviews and visits to regions (min. 10 half-days), and iv) reproduction costs of 50 booklets.
MoLSS and target institutions will provide appropriate furnished office space for the experts for the
period of their stay in Ankara and related running costs (telephone and fax, excluding international
telephone). MoLSS and relevant institutions will provide adequate staff time and timely and easy
access to all pertinent information requested by the Framework Contract Team, possibly in English
language. Target institutions will be notified through official letter about the visit of the Consultant and
office space will be requested.
The Consultant shall ensure that experts are adequately supported and equipped. In particular it shall
ensure that there is sufficient administrative, secretarial and interpreting provision to enable experts to
concentrate on their primary responsibilities. It must also transfer funds as necessary to support its
activities under the contract. The Framework Contract experts will be required to bring their own
computers/lap tops. Eventual communication (international phone calls), photocopy costs of
programme evaluation materials shall be covered by the Consultant.
283
6.7. For riders only: operational conditionality for intermediary payment if foreseen as per
article 7.2 b) of the Special Conditions
No interim payment is foreseen. An interim payment may be foreseen in the rider to a Specific
Contract if at least one of the conditions identified in the article 7.2 (b) of the Special Conditions is
fulfilled.
6.8. Others
The assignment will be implemented and supervised by Turkish Ministry of Labour and Social Security
- EU Coordination Department as the Operating Structure and the Beneficiary. Also, it is responsible
for the overall evaluation of the HRD OP.
An Internal Coordination Team, composed of 4 experts/assistant experts, will be formed within the OS
and it will be functioning separately from the management and control system. The Internal
Coordination Team is planned to be the direct counterpart of the Consultant and it cooperates and
ensures the preparation and delivery of the relevant documentation to be used during the
implementation of the assignment. The Consultant and Internal Coordination Team will meet once a
week in the Coordination Meetings to discuss issues related to the implementation of the assignment.
The Consultant and Internal Coordination Team may come together when needed for smooth ongoing
of the assignment.
Evaluation Sub-Committee (ESC) established under the SMC of the HRD OP, shall transmit its
comments on any guidance document about evaluation and monitoring systems; support the
independence of the evaluation function; and review and check the quality of works of the Consultant
(using an agreed quality check list) including inception and draft evaluation reports and issue
comments. The OS should compile all relevant comments provided by the ESC about the
reports/outputs of the Consultant and submit them in a clearly defined structure and timetable.
Senior Representative of Operation Beneficiary (SROB) will act as the official representative of the
MoLSS for this assignment and will bear overall responsibility for successful implementation of the
assignment through cooperating with the Consultant and ensuring that the team performs its tasks in
accordance with the pre-defined deadlines and the standard of quality required. He will formally
approve the reports and the technical documents. He will ensure the coordination for the development
and proper implementation of the assignment.
The Central Finance and Contracts Unit (CFCU) is the Contracting Authority and will be responsible
for all procedural aspects of the tendering process, contracting matters and financial management,
including payment of contract activities.
Contact Information
Contact person at the CFCU
Mr. Muhsin Altun
ProgrammeAuthorising Officer – CFCU Director
Tel: +90 312 295 49 00
Fax: +90 312 286 70 72
E-Mail: pao@cfcu.gov.tr
Address: EskiєehirYolu 4. km. 2.Cad. (Halkbank Kampьsь) No: 63 C-Blok 06580
Sцрьtцzь/Ankara Tьrkiye
Publicity and Visibility
The Consultant shall take all necessary measures to publicize the fact that the European Union has
financed the Program.
In addition, the Consultant shall take the necessary measures to ensure the visibility of the European
Union financing or co financing. These measures must comply with the rules laid down and published
by
the
Commission
on
the
visibility
of
external
operations:
http://ec.europa.eu/europeaid/work/visibility/index_en.htm
284
All projects /contract implemented under this programme shall comply with the Visibility Guidelines
for European Commission Projects in Turkey published by the EU Delegation to Turkey, at
http://www.avrupa.info.tr/AB_Mali_Destegi/Gorunurluk,Visi.html
All communication and visibility activities should be carried out in close co-operation with the CFCU.
The CFCU is the main authority in charge of reviewing and approving visibility-related materials and
activities. Before initiating any information, communication or visibility material and activity,
consultants and implementing partners should seek the approval of the CFCU in writing.
The EU-Turkey cooperation logo should be accompanied by the following text:
“This project is co-financed by the European Union and the Republic of Turkey.”
Whether used in the form of the EU-Turkey cooperation logo for information materials or separately at
events, the EU and Turkish flag have to enjoy at least double prominence each, both in terms of size
and placement in relation to other displayed logos and should appear on all materials and at all events
as per the Communication and Visibility Manual for European Union External Actions. At visibility
events, the Turkish and the EU flag have to be displayed prominently and separately from any logos.
Logos of the beneficiary institution and the CFCU should be clearly separated from the EU-Turkey
partnership logo and be maximum half the size of each flag. The logos will not be accompanied by any
text. The CFCU and beneficiary logo will be on the lower left-hand corner and lower right-hand corner
respectively. The consultant logo with the same size will be in the middle of the CFCU and beneficiary
logo. If the consultant is a consortium, only the logo of the consortium leader will be displayed.
Any publication by the Consultant, in whatever form and by whatever medium, including the Internet,
shall carry the following or a similar warning: “This document has been produced with the financial
assistance of the European Union”. In addition, the back cover of any such publications by the
Consultant should also contain the following disclaimer: “The contents of this publication is the sole
responsibility of name of the author/Consultant/implementing partner – and can in no way be taken to
reflect the views of the European Union”.
Taxand VAT Arrangements
The contract is, as a rule, exempt from all taxes and duties, including value added tax (VAT) and
Special Consumption Tax (SCT), Motor Vehicle Tax, Special Communication Tax, and/or taxes of
equivalent effect, stamp or registration duties or any other charge having equivalent effect, pursuant to
the provisions of Articles set out in the Framework Agreement 88 signed between Turkey and the EC in
2008, extract of Article 25 & 26 of which are provided in Annex I.
The Contractor shall accordingly complete the necessary formalities with the relevant authorities to
ensure that the goods and services required for performance of the Contract are exempt from taxes,
customs, import duties, levies and/or taxes of equivalent effect, and stamp or registration duties or any
other charge having equivalent effect.
Annexes to the Terms of Reference:
ANNEX I: Tax and Customs Arrangements
ANNEX II: Evaluation Grid
88
This is an extract of Article 25 and 26 of the IPA Framework Agreement signed between Turkey and the EC on
11.07.2008, and adopted as law by Turkish Parliament (no: 5824) on 03 December 2008 and which was
published in the Turkish Official Gazette on 07 December 2008, no: 27077. It has been put into force by the
Government Decree, no 2008/14450 that was published in the Official Gazette no: 27090 (Complementary Issue),
dated 24 December 2008. Please, also refer to the Framework Agreement and the Communiqués issued by the
Ministry of Finance (the MoF) thereto for further information, especially for exemption scope and implementation
procedure, which are available at the MoF’s website at: http://www.gib.gov.tr/index.php?id=1055.
285
Annex B3. Generic Question Frame for meetings
1. How does the HRD OP impact on work planning within the organisation
Do you make annual work plans for your organisation?
Have you assigned certain tasks to certain specific employees?
Is HRD OP management part of the overall work plan?
2. How many projects are related to your organisation / institution?
Can you tell me, briefly, about them?
Can you, briefly, tell me the process from tender through implementation and monitoring to final end
evaluation / what is your duties during the different stages?
Note: these questions can be checked through documents; the point is to obtain an understanding of
the stakeholders’ perception of their own roles
3. How does the HRD OP impact on daily routines and the organisation of work
4. How do you organise the administration of HRD OP?
Is the management of HRD OP an administrative burden?
Have you any ideas to how it could be done less bureaucratic and more smoothly?
5. How appropriate are the management structures formed at the levels of Operation Structure and
Operation Beneficiaries? What could be done differently in the whole system? What could be done
differently in your institution?
6. What are your direct partner institutions (to whom you report, who report to you, with whom you
communicate, to whom you send or from whom you receive documents according to formal document
flows and procedures)? How would you assess cooperation with each of the identified partner
institutions? Please describe any experiences/practices with other institutions that you would like to
change and what would you like to change?
7. How is the priory axes impact on your daily work
1) attract and retain more people in employment (women, young)
2) investment in human capital through education and VET (especially for women/girls)
3) promoting life-long learning
4) integration of the disadvantaged persons to the labour market
5) technical assistance
8. How do the HRD OP and priority axes meet?
9. Do the projects within the HRD OP mirror the local needs?
How do you assess local needs?
10. What sort of routines is used for follow-up procedures, for monitoring and evaluation, and for
registration (see and observe)
11. How is the communication between your organisation and the Ministry / central level?
12. To which extent is registration and monitoring procedures done in paper / PC / national MIS (see
QF for Information Systems)
286
13. To which extent is the data and information stream digitalised?
14. What sort of quality assurance procedures for data and information collected through the system?
15. Can we trust data and information collected? From your organisation? From local level?
16. In general, how do you see the formal procedures in HRD OP?
Is your main contact to the projects formal or non-formal – and which of the two serves your purpose
best?
(try to provide examples of informal communication)
17. Have you experienced any problems in finding and in visitation of participants in projects (for
instance training programmes etc.)?
- women
- young
- disabled
18. How are the facilities for disabled / women?
Do you perform on-site visits and controls?
How do you perform that (on-site visits and controls)?
19. How is the regional distribution of participants?
Are they just from the major cities and growth centres?
20. How is the dialogue with employers?
How is the outcome of the HRD OP (e.g. trainings) perceived by the local employers?
a. Employers participating in the programme
b. Employers NOT participating in the programme
Is there a risk of distortion of the local labour market due to the programme?
21. How well does the HRD OP meet the horizontal issues:
 equal opportunities for men and women;
 sustainable development & environmental protection;
 participation of civil society;
 geographic, sectoral and thematic concentration;
 concerns of disadvantaged persons;
 good governance
E.g.:
How does civil society impact on the HRD OP (and vice versa)? Does the civil society participate?
Who is the civil society according to your opinion?
What is good governance?
COOPERATION, SOCIAL DIALOGUE AND PARTNERSHIP APPROACH
22. How do you assess the cooperation between your institution and:
Employers
Training schools and educational centres
Chambers
Local authorities
MoLSS
287
İŞKUR
MoNE
23. Do your organisation take part in defining priorities, indicators, targets, and roles – or does the
initiative mainly originates from other (e.g. central organisations, ministries, etc.)
24. How does the partnership approach work on local and regional level?
SWOT
25. What do you consider to be your organisation’s / institution’s main strengths in programming and
implementing the HRD OP?
Do you use your strength or are there things you could have done more efficiently?
26. What do you consider to be your organisation’s / institution’s main Weaknesses in programming
and implementing the HRD OP?
How do your weaknesses impact on the HRD OP implementation?
27. What do you consider to be your organisation’s / institution’s main Opportunities in programming
and implementing the HRD OP?
How can the opportunities come into effective use for the forthcoming programme?
28. What do you consider to be your organisation’s / institution’s main Threats in programming and
implementing the HRD OP?
How do the threats impact on the HRD OP implementation?
288
Annex B4 Basic Survey on the functionality of the HRD OP:
Questionnaire
Page 1
of the HRD OP: Basic
Important note: The Interim Evaluation Team guarantees anonymity. All information entered in this
section is only meant to ensure that the survey covers all sectors and participants in the HRD OP. The
information only serves internal purposes on the Interim Evaluation Team and will be removed from
our report.
Evaluation
1. Please enter the name of your organisation
 European Commission or similar including
• National IPA coordinator
• EUD
• EUSG
• Other
 Ministry of Labour and Social Security (MoLSS)
 Turkish Employment Agency (İŞKUR)
• İŞKUR Central level (Ankara)
• İŞKUR Regional level
• İŞKUR Local level
 Ministry of National Education, MoNE, or related, including:
• The Council of Higher Education (YÖK)
• Vocational Qualification Authority (MYK)
• University
• Public Training Institutions involved in initial vocation or technical training (VET/TVET)
• Public Training Institutions involved in Formal Education
• Public Training Institutions involved in Adult Education (LifeLong
 Learning)
 Private Training Institutions; please specify your involvement in HRD OP:
• Initial vocation or technical training (VET/TVET)
• Formal Education
• Adult Education (LifeLong Learning)
 Ministry of Development / State Planning Organization (DPT)
• Central level (Ministry of Development)
• Regional Development Agency
 State Organization for the Development of Small and Medium Sized Businesses (KOSGEB)
 Small and Medium Sized Enterprises (KOBİ)
 Turkish Statistical Institute (TÜİK)
Page 2
Evaluation of the HRD OP: Basic
2. Please specify your type of organisation
 Governorship
 NGO/ Occupational Organization and Chambers; please specify:
• Chamber of Commerce and Industry
• Exporters’ Associations
• Trade union
• Employer association
 Other (please specify)
289
Page 3
Evaluation of the HRD OP: Basic
3. Please specify your title / seniority
4. Please specify your main task connected to HRD OP (please tick your prime task)
 SMC Member, including subcommittee
 Monitoring
 Selection of projects
 Inspection of projects
 Other (please specify)
5. Please specify your region
TR 00 Central level (e.g. Ministry or central organisation in e.g. Ankara or Istanbul)
TR 83 Samsun, Tokat, Çorum, Amasya
TR 72 Kayseri, Sivas, Yozgat
TR 90 Trabzon, Ordu, Rize, Giresun, Artvin, Gümüşhane
TR B2 Van, Muş, Bitlis, Hakkari
TR B1 Malatya, Elazığ, Bingöl, Tunceli
TR 63 Hatay, Kahramanmaraş, Osmaniye
TR C2 Şanlıurfa, Diyarbakır
TR A1 Erzurum, Erzincan, Bayburt
TR 82 Kastamonu, Çankırı, Sinop
TR C3 Mardin, Batman, Şırnak, Siirt
TR C1 Gaziantep, Adıyaman, Kilis
TR A2 Ağrı, Kars, Iğdır, Ardahan
6. Please indicate your Municipality
Page 4
Evaluation of the HRD OP: Basic
7. To what extent do the measures mirror the local and regional needs in your region?
 Criteria:
o Agree
o Moderately Agree
o Neutral
o Disagree
o I don’t know
 Question:
o Promoting women’s employment
o Promoting youth employment
o Promoting registered employment
o Increasing quality of public employment services
o Supporting girls’ enrolment and education
o Strengthening Vocational and Technical Education
o Strengthening links between Lifelong Learning, education system and labour market
o Increasing adaptability of employees and employers n
o Increasing integration of disadvantaged persons to the labour market
290
Page 5
Evaluation of the HRD OP: Basic
8. Do the selected HRD OP projects in your region reflect and address the local and regional needs?
 Criteria:
o Yes, to large extent
o Yes, to some extent
o Not really, we would like to add more
o Not at all
o I don’t know
 Questions:
o Promoting women’s employment
o Promoting youth employment
o Promoting registered employment
o Increasing quality of public employment services
o Supporting girls’ enrolment and education
o Strengthening Vocational and Technical Education
o Strengthening links between Lifelong Learning, education system and labour market
o Increasing adaptability of employees and employers
o Increasing integration of disadvantaged persons to the labour market
Page 6
Evaluation of the HRD OP: Basic
9. Do you find reason to revise any of the measures mention in the aforementioned questions perhaps
subject to the situation within your region?
 Criteria:
o YES
o NO
o I don’t know
 Questions:
o Promoting women’s employment
o Promoting youth employment
o Promoting registered employment
o Increasing quality of public employment services
o Supporting girls’ enrolment and education
o Strengthening Vocational and Technical Education
o Strengthening links between Lifelong Learning, education system and labour market
o Increasing adaptability of employees and employers
o Increasing integration of disadvantaged persons to the labour market
10. In case you typed “YES” in question 9 could you please specify why?
 Criteria:
o Not applicable to our region
o Any measure distorts the labour market
o Other (specify below)
 Questions:
o Promoting women’s employment
o Promoting youth employment
o Promoting registered employment
o Increasing quality of public employment services
o Supporting girls’ enrolment and education
291
o
o
o
o
o
Strengthening Vocational and Technical Education
Strengthening links between Lifelong Learning, education system and labour market
Increasing adaptability of employees and employers
Increasing integration of disadvantaged persons to the labour market
Other (please specify)
Page 7
Evaluation of the HRD OP: Basic
To what extent will the Programme meet its targets?
11. How effective are the evaluation and selection criteria?
Please state your opinion on the statements below.
 Criteria
o Agree
o No opinion
o Disagree
o I do not know
 Questions:
o Evaluation and selection criteria are objective
o The evaluation and selection criteria are fair or at least the only doable
o The evaluation and selection criteria ensure that the most successful projects are
selected
o The evaluation and selection criteria ensure that only projects which makes an impact
on the local development are selected
o The evaluation and selection criteria ensure a balance in number of projects across
priorities axis within the regions
o The evaluation and selection criteria ensure a fair balance in number of projects
across regions in accordance to regional capacity and needs
o The evaluation and selection criteria ensure that only projects which sufficiently
address the local and regional needs are selected
12. Please indicate how the evaluation and selection criteria can be improved in case you disagree
with any of the above statements:
Page 8
Evaluation of the HRD OP: Basic
13. How effective and balanced are the evaluation and selection of projects?
Please state your opinion on the statements below.
 Criteria
o Agree
o No opinion
o Disagree
o I do not know
 Questions:
o The selection of projects are done on an objective basis
o The selection of projects are balanced across priorities
o The selection of projects mirror the regional and local needs
o The selection of projects are balanced across regions
o The selection of projects are balanced within the 12 NUTS 2 regions
292
o
o
The selection of projects mirrors the need of less developed local areas within the
NUTS 2 regions
Regional stakeholders have sufficient influence on the selection of projects
Page 9
Evaluation of the HRD OP: Basic
14. Please consider the following statements
 Criteria
o Agree
o No opinion
o Disagree
o I do not know
 Questions:
o Potential beneficiaries of the Programme have enough information on project
o evaluation and selection criteria.
o Potential beneficiaries of the Programme have enough information on requirements
related to improvement of PTDs for final approval.
o High number of project proposals received after the first call for tender (2 nd Package)
has made it difficult to evaluate proposals properly.
o Project evaluation and selection process is effective enough (for instance, time spent
compared with the results).
o You have enough information on Programme monitoring indicators
o Programme indicators reflect Programme outputs/results
o Programme has effects that are not put forward by indicators
o Potential beneficiaries of the Program have been informed sufficiently about the
opportunities and benefits of the Programme.
o Potential beneficiaries of the Program have been informed sufficiently about project
selection.
o Potential beneficiaries of the Program have been informed sufficiently about
evaluation process.
o Potential beneficiaries of the Program have been informed sufficiently about
developing projects.
o Potential beneficiaries of the Program have been informed sufficiently about tendering
and implementation.
15. How could the selection of projects be improved?
Page 10
Evaluation of the HRD OP: Basic
16. Please indicate your view of the Operational Programme and its projects:
 Criteria:
o To a large extent
o To some extent
o Medium
o To a less degree
o Not at all
o I do not know
293

Questions:
o To what extent do you think the projects implemented in your region have increased
women employment?
o To what extent do you think the projects implemented in your region have increased
youth employment?
o To what extent do you think the projects implemented in your region have increased
registered employment?
o To what extent do you think the projects implemented in your region have
strengthened public employment offices?
o To what extent do you think the projects implemented in your region have increased
girls’ enrolment rate?
o To what extent do you think the projects implemented in your region have
strengthened vocational and technical education?
o To what extent do you think the projects implemented in your region have
strengthened links between education system and labour market?
o To what extent do you think the projects implemented in your region have increased
adaptability of employees and employers?
o To what extent do you think the projects implemented in your region have contributed
to integration of disadvantaged groups to labour market?
Page 11
Evaluation of the HRD OP: Basic
17. Please indicate your view on the functioning of the Regional Grant Monitoring
Teams
 Criteria
o Agree
o No opinion
o Disagree
o I do not know
 Questions:
o Responsibilities of Regional Grant Monitoring Teams, RGMTT, regarding the
management and implementation of HRD OP have been defined clearly.
o Regional Grant Monitoring Teams, RGMTT, are effective and sufficient
o Rotation principle ensures Regional Grant Monitoring Teams, RGMTT, to be effective
and efficient.
o The time allocated to meet the task of the Regional Grant Monitoring Teams, RGMTT,
are sufficient and reasonably
o The Regional Grant Monitoring Teams, RGMTT, ensure effective support and backup
for the projects
o The Regional Grant Monitoring Teams, RGMTT, ensure that the projects runs
effectively
o The Regional Grant Monitoring Teams, RGMTT, ensure proper and timely inspection
of the projects in accordance with their monitoring role
18. How could the functioning of the Regional Grant Monitoring Teams be improved?
294
Page 12
Evaluation of the HRD OP: Basic
19. Please indicate your view on the functioning of the Sectoral Monitoring Committee
 Criteria
o Agree
o No opinion
o Disagree
o I do not know
 Questions:
o Responsibilities of Sectoral Monitoring Committee Members regarding the
management and implementation of HRD OP have been defined clearly.
o SMC meetings are effective and sufficient
o Rotation principle ensures SMC meetings to be effective and efficient
o Time allocated to discuss agenda during SMC meetings is enough.
20. How could the functioning of the Sectoral Monitoring Committee be improved?
Page 13
Evaluation of the HRD OP: Basic
21. To what extent do the achieved results correspond to the resources spent (financial
resources, human resources, time)? Please consider the following statements:
 Criteria
o Agree
o No opinion
o Disagree
o I do not know
 Questions:
o Human resources of MoLSS EU Coordination Department, EU Turkey Delegation,
Central Finance and Contracts Unit and beneficiary organizations is sufficient in terms
of quality to fulfil tasks related to program management and implementation.
o Human resources of MoLSS EU Coordination Department, EU Turkey Delegation,
Central Finance and Contracts Unit and beneficiary organizations is sufficient in terms
of quantity to fulfil tasks related to program management and implementation.
22. According to you, what are the fundamental success factors (internal and external)
which determine the efficiency of HRD OP up until now? Please state your opinion to
the box below by using key words or short sentences.
23. According to you what are the fundamental problematic areas (internal and external)
which determines the efficiency of the HRD OP up until now? Please state your opinion
to the box below by using key words or short sentences.
Page 14
Evaluation of the HRD OP: Basic
This constitutes the end of the online evaluation questionnaire.
THANK YOU FOR YOUR KIND PARTICIPATION!
295
Annex B5 Survey on Indicators: Questionnaire
Page 1
Evaluation of the HRD OP: Indicators
This Questionnaire serves to obtain some brief opinions on the relevance and the effectiveness of the
indicators selected for monitoring and evaluation of the HRD OP.
Background information:
The HRD OP operates with two types of indicators: Result and Output Indicators.
Results indicators serve to measure both Outcome and Impact (and not least net impact), while output
indicators measures output, which is the basic result from undertaken a specific action (e.g. 20
trainees trained). The HRD OP does not explicitly define input indicators.
In here we have copied the Result indicators as they are mentioned in the programme.
We do not ask you to comment on the theory of evaluation and monitoring purposes. However, we
kindly ask you to provide some few comments on the relevance of the indicator subject to the Measure
and the Objective.
Page 2
Evaluation of the HRD OP: Basic
Important note: The Interim Evaluation Team guarantees anonymity. All information entered in this
section is only meant to ensure that the survey covers all sectors and participants in the HRD OP. The
information only serves internal purposes on the Interim Evaluation Team and will be removed from
our report.
1. Please enter the name of your organisation
 European Commission or similar including
• National IPA coordinator
• EUD
• EUSG
• Other
 Ministry of Labour and Social Security (MoLSS)
 Turkish Employment Agency (İŞKUR)
• İŞKUR Central level (Ankara)
• İŞKUR Regional level
• İŞKUR Local level
 Ministry of National Education, MoNE, or related, including:
• The Council of Higher Education (YÖK)
• Vocational Qualification Authority (MYK)
• University
• Public Training Institutions involved in initial vocation or technical training (VET/TVET)
• Public Training Institutions involved in Formal Education
• Public Training Institutions involved in Adult Education (LifeLong
 Learning)
 Private Training Institutions; please specify your involvement in HRD OP:
• Initial vocation or technical training (VET/TVET)
• Formal Education
• Adult Education (LifeLong Learning)
296




Ministry of Development / State Planning Organization (DPT)
• Central level (Ministry of Development)
• Regional Development Agency
State Organization for the Development of Small and Medium Sized Businesses (KOSGEB)
Small and Medium Sized Enterprises (KOBİ)
Turkish Statistical Institute (TÜİK)
Page 3
Evaluation of the HRD OP: Basic
2. Please specify your type of organisation
 Governorship
 NGO/ Occupational Organization and Chambers; please specify:
• Chamber of Commerce and Industry
• Exporters’ Associations
• Trade union
• Employer association
 Other (please specify)
Page 4
Evaluation of the HRD OP: Basic
3. Please specify your title / seniority
4. Please specify your main task connected to HRD OP (please tick your prime task)
 SMC Member, including subcommittee
 Monitoring
 Selection of projects
 Inspection of projects
 Other (please specify)
5. Please specify your region
TR 00 Central level (e.g. Ministry or central organisation in e.g. Ankara or Istanbul)
TR 83 Samsun, Tokat, Çorum, Amasya
TR 72 Kayseri, Sivas, Yozgat
TR 90 Trabzon, Ordu, Rize, Giresun, Artvin, Gümüşhane
TR B2 Van, Muş, Bitlis, Hakkari
TR B1 Malatya, Elazığ, Bingöl, Tunceli
TR 63 Hatay, Kahramanmaraş, Osmaniye
TR C2 Şanlıurfa, Diyarbakır
TR A1 Erzurum, Erzincan, Bayburt
TR 82 Kastamonu, Çankırı, Sinop
TR C3 Mardin, Batman, Şırnak, Siirt
TR C1 Gaziantep, Adıyaman, Kilis
TR A2 Ağrı, Kars, Iğdır, Ardahan
6. Please indicate your Municipality
297
Page 5
Evaluation of the HRD OP: Indicators
Please have a look at the measures and the specific objectives defined in the HRD OP and the
following indicators which are meant to measure the outcome of the measures and steps taken. The
sections are divided into the five priority axes. The questionnaire mentions briefly their measures and
their so-called “specific objectives”. From here, and kindly asking you to consider the measure and
objectives, we kindly ask you to have a view on the indicators.
Our question is: Are these indicators relevant, are data / information achievable, and can the
indicators be used in the future after the HRD OP has been concluded?
Page 6
Evaluation of the HRD OP: Indicators
Priority axis 1:
To attract and retain more people in employment, particularly by increasing labour force participation
of women, and decrease unemployment rates especially for young people
7. Assuming the Specific objective 1: Promote labour force participation and employment of women,
including those formerly employed in agriculture. Are the following indicators relevant for the
objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increased rate of women having entered into labour market (having the qualification
and motivation to seek a job) after participating in the courses or counselling
o Increased rate of women employed through the courses and counselling
o Increased rate of women entrepreneurs
8. If no: What is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
298
Page 7
Evaluation of the HRD OP: Indicators
9. Assuming Specific objective 2: To increase youth employment.
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increased rate of young people employed following the courses, internship and
counselling activities
o Increased rate of young entrepreneurs
10. If no: What is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 8
Evaluation of the HRD OP: Indicators
11. Assume Specific objective 3: To promote registered employment
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Decrease in the rate of complaints (advice slip) on the unregistered employment to
the Insurance Inspection Board
o Increase in the number of bilateral projects on promoting registered employment
between the relevant partners
o Increase in the rate of registered people in 43 city centres through the bilateral
projects and local partnerships
12. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
299








Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Page 9
Evaluation of the HRD OP: Indicators
13. Assume Specific objective 4: To improve public employment services.
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increase in the number of unemployed people counselled
o Increase in the placement rates of İŞKUR
14. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 10
Evaluation of the HRD OP: Indicators
Next consider Priority axis 2:
To enhance investment in human capital by increasing the quality of education, improving the linkage
between education and labour market, and raising enrolment rates at all levels of education, especially
for girls
15. Assume Specific objective 1: To increase enrolment rates particularly for girls at all
levels of education and vocational training.
Are the following indicators relevant for the objective?
300


Criteria:
o YES
o NO
o Do not know
Questions:
o Increased percentage of girls accessing to education, particularly secondary
education, as a result of programme intervention
o Reduction in the number of girls dropping out of secondary education and VET
o Increase in the number of girls graduating from secondary education and VET
16. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 11
17. Assume Specific objective 2: To increase the quality of education especially in
vocational education and training.
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o The level of satisfaction of teachers, students and entrepreneurs who participated in
the activities regarding the content and quality of Vocational and Technical Education.
o Increasing demands for 45 pilot vocational schools.
o Teacher training in line with the newly updated curriculum.
18. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
301




The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Page 12
Evaluation of the HRD OP: Indicators
19. Assume Specific objective 3: To develop innovative approaches to improve linkage between
education and labour market
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o No Indicators are available
20. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 13
21. Assume Specific objective 4: To promote a “partnership approach” to modernize VET system
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o No Indicators are available
22. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
302









Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Page 14
Priority axis 3:
To promote a “partnership approach” to modernize VET system.
23. Assume Specific objective 1: To promote Life Long Learning (LLL) Opportunities under a LLL
Strategy.
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Share of primary education and secondary education graduates among participants
certified and tested in terms of vocational knowledge and skills in 10 VOCTEST
Centres
o Increased percentage of participants who obtained a certificate in vocational
knowledge and skills
o Increased percentage of participants of participants who obtained a certificate in basic
skills
o Increased percentage of those who found work after completing an employability
course
Page 15
24. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
303
Page 16
25. Assume Specific objective 2: To improve quality of non-formal trainings
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o No Indicators are available
26. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 17
27. Assume Specific objective 3: To increase adaptability of employees
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increased adaptability of workers in terms of having extra skills or abilities related to
their position (percentage of participants increasing their salary or being promoted or
having extra responsibilities)
o Increased training activities on the requirements of adaptability given by social
partners to their members
28. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
304






Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Page 18
29. Assume Specific objective 4: To increase adaptability of employers in SMEs
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increased adaptability of employers in terms of having extra skills or abilities in order
to be competitive enough (percentage of participants increasing number of workers,
change of the branch of activity)
30. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 19
31. Assume Specific objective 5: To promote well-functioning of the National Qualifications System
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Number of certificates issued by VOCTEST Centres
32. If no, what is the problem?
305











Net impact cannot be measured since Impact can arise from other factors in the environment
Data is not available
Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
Data differs from region to region (not standardized)
Data are not systematically collected (e.g. timing differs between projects and/or regions)
Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
The indicator does not mirror the real target / is not relevant
The target group is not clearly defined
There is no baseline data (we cannot see and measure improvements)
The indicator is too complicated to calculate (if based upon aggregated data)
Other (please specify)
Page 20
Evaluation of the HRD OP: Indicators
Priority axis 4:
To promote an inclusive labour market with opportunities for disadvantaged people, with a view to
their sustainable integration into the labour force and combat all forms of discrimination in the labour
market.
33. Assume Specific objective 1: To facilitate sustainable integration of the disadvantaged into the
labour market.
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increased rate of disadvantaged persons entered into labour market (having the
qualification and motivation to seek a job) following the rehabilitation programmes,
courses and counselling.
o Increased rate of disadvantaged persons employed following the courses and
counselling services
34. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
306
Page 21
35. Assume Specific objective 2: To improve the functioning and coordination of institutions and
mechanisms in the field of labour market and social protection, particularly in order to facilitate the
integration of disadvantaged persons into the labour market.
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o Increased satisfaction on the services both in institutional and target group level (by
staff of institutions and by disadvantaged persons)
36. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 22
Evaluation of the HRD OP: Indicators
Priority axis 5:
Technical Assistance
37. Assume Specific objective 1: Strengthening the necessary capacity at central level to efficiently
develop, implement, evaluate, monitor and control the IPA funds in the period 2007-2009 within the
framework of HRD OP and improving the information about the ESF structures and best examples,
providing assistance in the transition to Decentralised Management without ex-ante control of the EC
Delegation
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o No Indicators are available
307
38. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
Page 23
Evaluation of the HRD OP: Indicators
39. Assume Specific objective 2: Increasing the absorption capacity of the final beneficiaries and
administrative capacity of stakeholders which may get more responsibilities in the upcoming period
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o No Indicators are available
40. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
308
Page 24
41. Assume Specific objective 3: Increasing the information and public awareness with respect to the
preparation for and effective use of IPA funds in Turkey in line with the HRD OP
Are the following indicators relevant for the objective?
 Criteria:
o YES
o NO
o Do not know
 Questions:
o No Indicators are available
42. If no, what is the problem?
 Net impact cannot be measured since Impact can arise from other factors in the environment
 Data is not available
 Data is difficult to achieve (e.g. not digitalised, only available in paper, only by submission of
reports, etc.)
 Data differs from region to region (not standardized)
 Data are not systematically collected (e.g. timing differs between projects and/or regions)
 Data is only generated from on-going projects (we cannot see what status are in locations
without projects, before the project is launched, and after the project has been concluded)
 The indicator does not mirror the real target / is not relevant
 The target group is not clearly defined
 There is no baseline data (we cannot see and measure improvements)
 The indicator is too complicated to calculate (if based upon aggregated data)
 Other (please specify)
This constitutes the end of the online evaluation questionnaire on indicators.
THANK YOU FOR YOUR KIND PARTICIPATION!
309
The content of this publication is the sole responsibility of the Consultant and can in no way be
taken to reflect the views of the European Union.
The content of this publication is the sole responsibility of the Consultant and can in no way be
310
taken to reflect the views of the European Union.