PISA for Development
Initial Technical Meeting
Overview Presentation
27 – 28 June 2013
Paris, France
EDU/DCD
PISA for Development
Initial Technical Meeting
Expected Results from Meeting
1. Shared understanding among participants and
partners
2. A general agreement regarding the main
technical challenges to be addressed
3. Framework established for the working methods
and focus of the technical partnerships
4. Proposals presented for membership of the Steering
Group, the Technical Advisory Group, and for the
first set of technical papers to be commissioned
2
Overview of agenda and sessions: Day
One
• PISA for development: presentation of the
project
• Roundtable on country and development
partner perspectives
• The experience of Brazil in PISA
• Stock-take of main technical challenges
3
Overview of agenda and sessions: Day
Two
• Review of main technical challenges
• Building on existing work
• Framework for working methods and
technical partnerships
• Next steps and meeting conclusion
4
Key documents in folder
Annotated agenda
Participants list
Final version of project document
Draft ToR for International Steering Group
Draft ToR for Technical Advisory Group
Draft Agreement for participation in PISA for
development
• Draft roles and responsibilities expected of
National Centres
•
•
•
•
•
•
5
PISA for Development
•Erik Solheim – Chairman of the Development Assistance
Committee
•Andreas Schleicher – Deputy Director, Directorate for
Education and Skills
PISA for Development
Initial Technical Meeting
Motivation for PISA for development
An ambition to make the benefits of
PISA available to a broader group of
countries
An opportunity to support the measurement of a post2015 education goal that is focused on learning quality
• An aim to enable all of the countries of the global
community to be on a single metric with regards to key
educational outcomes
8
Benefits of participation 1:
being part of ..............
• The international education community
that aims to improve student outcomes
globally through research;
• In-depth comparative analyses of factors
that are strongly related to student
outcomes;
• National and international debates about
how to improve student outcomes;
9
Benefits of participation 2: good
practices and peer learning
• Participants in PISA are exposed to worldclass assessments and rigorous international
standards.
• PISA allows countries to learn policy lessons
from other countries, particularly from those
that may share common challenges and
conditions, thereby facilitating peer learning
and the dissemination of good practices and
knowledge of what works to improve student
outcomes
10
Benefits of participation 3: policy/system
reform impacts
• Focus on student learning for policy
reforms
• Focus on teacher professional
development and training
• School leadership and school
autonomy
• Focus on standards that are congruent
and aligned
• Focus on student competencies and
skills beyond (curricular) content
11
PISA for Development
Enhancing the relevance and use of PISA
… in support of participating countries
Expected impact
Increased numbers of developing countries use
PISA assessments from 2016 onwards to:

Monitor progress towards national improvement
targets

Comparatively analyse factors associated with
student outcomes

For institutional capacity-building, and

For tracking international education targets within a
post-2015 framework
12
PISA for Development
How we arrived at this point – the journey
• So far, PISA has covered 74 countries, 28 of which are
developing countries (ODA recipients)
• Our experience so far:
•
Relevance of the PISA approach and assessment
•
Adherence to technical standards
•
Quality of technical implementation matches that of OECD countries
•
Full compliance with confidentiality requirements
• Some open issues
•
Match between student ability distribution and item difficulties
•
Relevance of context questionnaires
•
Out-of-school populations
• How far can we go with expanding PISA as we know it
geographically?
PISA for Development
Benchmarking for improvement
Reading performance and national wealth in low income countries
600
Shanghai-China
PISA reading performance
550
Chinese Taipei
500
Turkey
450
400
350
Estonia
Hungary
Poland
Latvia
Czech Rep.
Croatia
Russia Lithuania
Slovak Rep.
Chile
Malta
Serbia Costa Rica
Bulgaria
Uruguay
Romania Mexico
Thailand
Mauritius
Colombia
Miranda-Venezuela
Malaysia
Montenegro
Brazil
Indonesia Tunisia Jordan
Trinidad and Tobago
Kazakhstan
Moldova Albania
Argentina
Georgia
Panama
Peru
Azerbaijan
Tamil Nadu
Linear (Non-high income countries (GDP < 20 000))
Himachal Pradesh
300 Kyrgyzstan
0
2000
Non-high income countries (GDP < 20 000)
4000
6000
8000
10000
12000
14000
16000
GDP per capita (USD converted using purchasing power parity)
18000
20000
14
PISA for Development
Benchmarking for improvement
600
Reading performance and national wealth
PISA reading performance
550
500
450
400
High income countries (GDP > 20 000)
Non-high income countries (GDP < 20 000)
350
Linear (High income countries (GDP > 20 000))
Linear (Non-high income countries (GDP < 20 000))
300
0
10000
20000
30000
40000
50000
60000
70000
80000
GDP per capita (USD converted using purchasing power parity)
90000
100000
15
Discussion with Partners: timeline
• Initial dialogue with the largest donors to
education and technical agencies, Summer
2012
• Meeting in November 2012 to discuss first
draft of Project Document
• Second draft of Project Document circulated
in March 2013 – initial commitments of
support from development partners and
technical agencies
• Dialogue with potential pilot countries
16
Dialogue with possible pilot countries
•
•
•
•
•
•
•
•
Cambodia
Ecuador
Guatemala
Mongolia
Punjab (Pakistan)
Senegal
Sri Lanka
Zambia
17
Dialogue with development partners
•
•
•
•
•
•
•
France
Germany (BMZ/GIZ)
IADB
Korea
Norway
UK (DFID)
World Bank
18
Dialogue with other agencies
•
•
•
•
•
•
•
•
PASEC
SACMEQ
UNESCO
UIS
UNICEF
EFA GMR
GPE
CUE, Brookings Institution
19
PISA for Development
Enhancing the relevance and use of PISA
• Aims: to enhance the policy relevance of
PISA for developing countries through:
– the development of enhanced PISA survey
instruments and data collection methods…..
– that are more relevant for the contexts found in
developing countries…..
– but which produce results on the same scales
as the main PISA assessment.
20
PISA for Development
Plans for the Pilot
… in support of developing countries
 Five (to seven) partner
countries
 Five main outputs
 Five project phases over
36 months
21
PISA for Development
Enhancing the relevance and use of PISA
Main Phases and Governance
36 Months of Implementation
I.
Design, Planning and Coordination
II.
Technical Development
III.
Field Trial and In-country Data
Collection
IV.
Analysis and Reporting
V.
Post-Pilot Governance
PISA for Development
Enhancing the relevance and use of PISA
Five Main Outputs
1. Contextual questionnaires and datacollection instruments enhanced (e.g. for
students, parents, schools, etc.): Examples:
• Socio-economic background of students and
schools
• School climate
• Resource availability and use
• Autonomy and accountability
• Governance
23
PISA for Development
Enhancing the relevance and use of PISA
Five Main Outputs
2. The descriptive power of cognitive
assessments enhanced in reading,
mathematics and science
•
Review, select, translate, validate and combine
into test booklets
•
Improved targeting at lower-end of proficiency
•
Component Reading Skills assessment
24
PISA for Development
Enhancing the relevance and use of PISA
Five Main Outputs
3. An approach developed, including a
methodology and analytical framework,
for including out-of-school 15 year-olds
in the assessments
•
Draw on existing work UNICEF, UNESCO
•
World Bank STEP project
•
PIAAC and others
25
PISA for Development
Enhancing the relevance and use of PISA
Five Main Outputs
4. Country capacity in assessment, analysis
and use of results for monitoring and
improvement strengthened among
participating countries
•
•
•
•
Planning process with each participant
Customised country-specific report for
participants
Working with development partners
Succession planning and knowledge transfer
26
PISA for Development
Enhancing the relevance and use of PISA
Five Main Outputs
5. Peer-to-peer learning opportunities
identified related to participation in PISA
• Experience of previous and current PISA
participants
• Seminar for pilot countries and others
• Contribute to UN-led post-2015 discussions
27
PISA for Development
Management and budget
Main Phases and Governance
Partnership with and guidance from 5 (to 7)
participating countries
International Steering Group
Technical Oversight
and Coordination
International and
National
Implementation
Overall budget (5 participating countries = Eur 2.9 million)
28
PISA for Development
Initial Technical Meeting
PISA for Development
Roundtable countries and development partners
• Main expectations for the meeting of
countries and development partners
• Perspectives of countries and
development partners on the project,
specifically:
 Country-specific opportunities and challenges
from participating in PISA
 Perspectives of development partners on the
opportunities and challenges
The PISA Instruments
Enrolment of 15-year-old students
Panama
Indonesia
Costa Rica
Turkey
These results from
Azerbaijan
PISA 2009 for nonColombia
OECD countries (and
Mexico
Mexico and Turkey)
Albania
show that among
Uruguay
PISA participants,
Brazil
there are still large
Kyrgyzstan
percentages of out-ofArgentina
school youth
Bulgaria
Thailand
Mauritius
Peru
Georgia
Russian Federation
Trinidad and Tobago
Malaysia
Partner average
OECD average
OECD average (excl. Turkey, Mexico, Chile)
0%
20%
40%
60%
80%
100%
31