International Advisory Group
Meeting
Closing Session
27-28 May 2014
OECD Secretariat
1

PISA FOR DEVELOPMENT

PISA for Development
International Advisory Group - Conclusions
Strong representation at the meeting from
• Countries signed up or committed to participation in
PISA for Development (Cambodia, Ecuador,
Guatemala, Rwanda, Senegal, Tanzania, Zambia),
• Development partners (France, Germany (BMZ/GIZ),
IADB, Ireland (Irish Aid), Japan(JICA),Korea,
Norway, UK (DFID), World Bank),
• International agencies (UNESCO, UIS, UNICEF, EFA
GMR, CONFEMEN and PASEC, LLECE), and
• Independent technicians in the field.
• Observers from Luxembourg and the United States

PISA for Development
International Advisory Group - Conclusions
• Reconfirmed unanimous support for the
PISA for Development project; and
• Acknowledged again the unique value that
PISA has as an international benchmarking tool for quality and equity in schooling and for guiding policies for system improvement.
• Appreciated the actions taken since June
2013
• Noted the indicative timeline and implementation schedule for the project and noted that this is subject to change as country schedules are established

PISA for Development
International Advisory Group - Conclusions
• Celebrated the signing of the Participation
Agreement between OECD and Ecuador and the remarks of Harvey Sanchez ,
Executive Director of INEVAL, who explained how participation in PISA for
Development will help drive up the quality and inclusiveness of the school system in
Ecuador.

PISA for Development
International Advisory Group - C onclusions
• Adopted the ToR for the IAG and the content of the Annual Report, subject to certain amendments, including clarifying:
– the respective roles and expectations of the constituent groups of the IAG
– the roles between the IAG and the TAG
– the overall governance of the project
• Noted that the IAG will have a chance to further review through written consultation
• Agreed to nominate co-chairs who would be elected at the next meeting of the IAG scheduled for March 2015
6

PISA for Development
International Advisory Group - Conclusions
• Endorsed the issues and challenges identified in the expert paper on enhancing the PISA cognitive instruments and
• Agreed on the options for adapting the cognitive instruments to fit the contexts of developing countries including in the four key aspects of review of assessment framework and items, test design, review of proficiency levels and review of scaling models.
• Noted that sampling and target population comparability was also an issue to seek bidders proposals on.
• Noted the need to ensure sufficient flexibility in order to serve national policy needs, while at the same time providing valid international benchmarking.

PISA for Development
International Advisory Group - Conclusions
• Endorsed the issues and challenges identified in the expert paper on enhancing the PISA context questionnaires; and
• Agreed on the options for adapting the questionnaires to fit the contexts of developing countries including measures of early learning opportunities, language at home and school, family and community support, quality of instruction, learning time, student socioeconomic status and school resources.
• Noted the constraint of the length of the student questionnaire and that national optional items could be included to meet specific national needs but suggested that a common core of questionnaire items be identified to permit international comparisons
• Discussed the potential value and resource implications of teacher and parent questionnaires and suggested that bidders are asked to comment on the value added of including these in
PISA for development.

PISA for Development
International Advisory Group - Conclusions
• Agreed the importance of seeking approaches for including out of school 15-year-olds in the study and recognised the significant challenges involved and the possible options for addressing these.
• Reviewed and endorsed the first draft of the expert paper on out of school 15 year olds and noted the next steps in discussing and finalising this by
October 2014.
• Recognised this as a distinct strand of the project and that this provided the opportunity for researching different approaches that may vary to meet country-specific needs.

PISA for Development
International Advisory Group Conclusions
Agreed on the draft principles and components of Terms of Reference for the International
Contractor(s) related to the cognitive instruments and contextual questionnaires with certain modifications, amplifications and additions, including:
• greater flexibility for bidders to respond to issues and questions and less prescription
• need to have a sufficient framework on which bids can be equally judged
• expert papers will be provided to bidders
10

PISA for Development
International Advisory Group Conclusions
• bidders should be open to working with other existing assessment organisations and programmes
• bidders should be asked to clarify the human and financial resource implications of their proposals
• bidders should make use of local contractors in all aspects of the work alongside the international contractor(s)
• ensure there are adequate national structures in place to effectively implement PISA for
Development
11

PISA for Development
International Advisory Group Conclusions
• For contextual questionnaires:
need to have clear evaluation criteria with an emphasis on cross-cultural validity
bidders should make use of and reference the literature that exists on factors affecting learning outcomes in each of the regions
12

PISA for Development
International Advisory Group Conclusions
• For contextual questionnaires:
bidders should propose the best respondents to target in order to get robust information on the key features
bidders should be asked to provide separately: (1) human and financial resource implications for student and school questionnaires and (2) human and financial resource implications for additional questionnaires that may be proposed
13

PISA for Development
International Advisory Group Conclusions
• For cognitive instruments, Secretariat to examine the potential for including in the pool of secure items those that were rejected for main PISA
Noted OECD’s plans for completing and consulting the IAG on the full ToR for the
International Contractor(s) and the tendering process
14

PISA for Development
International Advisory Group - Conclusions
• Noted the important role that PISA for
Development is going to play in building capacity in the participating countries, including
– for developing and conducting national and international large scale assessments, and
– for using performance data to diagnose strengths and weaknesses in the education system, and for supporting school improvement efforts.

PISA for Development
International Advisory Group - Conclusions
• Endorsed the plans for completing the capacity needs analysis and designing capacity building programmes for the participating countries
• Noted how the capacity building plans will be incorporated within the countries’ project implementation plans and coordinated with other strategies and support for assessment in the participating countries.
• Requested that the responsibilities of the
International Contractor(s) for capacity building are fully reflected in the ToR.

PISA for Development
International Advisory Group - Conclusions
• Noted and supported the plans for peerto-peer learning, especially the high level strategic report
• Noted the important role that PISA for
Development and PISA can play in contributing to and measuring post-2015 global educational targets that are focussed on learning.

PISA FOR DEVELOPMENT

Next Steps
• OECD to complete the remaining planned expert papers and hold third technical workshop (OOS)
• OECD to prepare ToR and tendering documents for international contractor(s) and to finalise these in consultation with the IAG members
• OECD to sign participation agreements with all the countries participating in the project
• Participating countries to nominate NCs and
NPMs and to prepare project implementation plans
19

Next steps continued
• Participating countries finalise agreements with development partners regarding contributions and support (e.g. international costs, in-country costs and activities)
• OECD to complete capacity needs analysis and to design capacity-building programmes for all participating countries
• OECD to finalise with development partners outstanding agreements for support to the project – general contributions and country-specific contributions
• OECD to confirm with other agencies arrangements for technical partnerships and collaboration
• First meeting of Technical Advisory Group after awarding of international contract
20

Next steps continued
• Complete tendering process to commission international contractor (s) by OECD
• Project implementation:
Design, planning and coordination (including capacity assessment and capacity building planning for participating countries);
Technical development
Field trials and main data collection;
Analysis and reporting
Post-pilot governance
21