Highlights from PISA
Slide 1
Highlights from the Program for International Student Assessment (PISA)
PISA 2012
Slide 2
What is PISA?
International large-scale assessment organized by the Organization for Economic Cooperation and Development (OECD)
NCES is U.S. coordinating and funding organization. PISA Governing Board (PBG) determines policy
Assessment of 15-year-old students
Administered every 3 years 2000
Slide 3
What is PISA?
In 2012, 68 education systems participated in PISA.
Albania, Argentina, Australia, Austria, Belgium, Brazil, Bulgaria, Canada, Chile, Colombia, Connecticut-USA, Costa Rica,
Croatia, Cyprus, Czech Republic, Denmark, Estonia, Finland, Florida-USA, France, Germany, Greece, Hong-Kong-CHN,
Hungary, Iceland, Indonesia, Ireland, Israel, Italy, Japan, Jordan, Kazakhstan, Korea, Latvia, Liechtenstein, Lithuania,
Luxembourg, Macao-CHN, Malaysia, Massachusetts-USA, Mexico, Montenegro, Netherlands, New Zealand, Norway,
Peru, Poland, Portugal, Qatar, Romania, Russian Federation, Serbia, Shanghai-CHN, Singapore, Slovak Republic, Slovenia,
Spain, Sweden, Switzerland, Taipei-CHN, Thailand, Tunisia, Turkey, United Arab Emirates, United Kingdom, United
States, Uruguay, and Vietnam.
Slide 4
What is PISA?
Assessment subjects: Mathematics, science, reading literacy. Mathematics was main subject assessed in 2012.
Computer-based mathematics and reading literacy. Optional assessment taken by 32 education systems.
Content knowledge, not limited to school-based curricula
PISA assesses applied knowledge/literacy: “How well can students nearing the end of compulsory schooling apply their
knowledge to real-life situations?”
Slide 5
What PISA reports
Average scores: Scale of 0-1000 for all domains
Proficiency levels: Percentages of students scoring at 6 levels
Trends: Change between average scores in 2012 and the scores in previous assessment years
Subgroup scores: International (e.g. gender, language spoken in home) and U.S. specific variables (e.g., race/ethnicity)
Slide 6
U.S. PISA Findings
1
Slide 7
General patterns of U.S. results
Ranked better in reading literacy than in mathematics and science literacy
Below OECD average score in mathematics only
Higher percentage at low proficiency levels than OECD average in mathematics only
Lower percentage of top performers than OECD average in mathematics only
No measurable change in average scores in mathematics, science, or reading literacy
Slide 8
Mathematics literacy
Slide 9
Quantity (25%): Are 15-year-olds able to comprehend multiple representations of numbers, engage in mental
calculation, employ estimation, and assess the reasonableness of results?
Uncertainty and Data (25%): Can students use probability and statistics and other techniques of data representation and
description to mathematically describe, model, and interpret uncertainty?
Space and Shape (25%): Can students understand perspective, create and read maps, and manipulate 3D objects?
Change and Relationships (25%): Can students model change and relationships with the appropriate functions and
equations?
Slide 10
PISA 2012 mathematics literacy content categories
Formulate (25%): Can 15-year-olds recognize and identify opportunities to use mathematics and then provide
mathematical structure to a problem presented in some contextualized form in order to formulate situations
mathematically?
Employ (25%): Are students able to employ mathematical concepts, facts, procedures, and reasoning to solve
mathematically-formulated problems and obtain mathematical conclusions?
Interpret (25%): Can students interpret, apply and evaluate mathematical outcomes in order to determine whether
results are reasonable and make sense in the context of the problem?
Slide 11
PISA mathematics literacy proficiency levels
Level 1: Identify information and carry out routine procedures according to direct instructions in explicit situations.
Level 2: Employ basic algorithms, formulae, procedures, or conventions. Capable of direct reasoning and making literal
interpretations of the results.
Level 3: Execute clearly described procedures, including those that require sequential decisions. Select and apply simple
problem-solving strategies.
Level 4: Work effectively with explicit models that may involve constraints or making assumptions. Capable of reasoning
with some insight in straightforward contexts.
Level 5: Work strategically using broad, well-developed thinking and reasoning skills, appropriate linked representations,
symbolic and formal characterizations and insight pertaining to these situations.
Level 6: Apply insight along with a mastery of symbolic and formal mathematical operations and relationships to develop
new approaches and strategies for attacking novel situations.
2
Slide 12
Mathematics literacy – Example item
Proficiency Level 1
Question: CHARTS
In January, the new CDs of the bands 4U2Rock and The Kicking Kangaroos were released. In February, the CDs of the
bands No One’s Darling and The Metalfolkies followed. The following graph shows the sale of the bands’ CDs from
January to June.
Jan: 4U2Rock – 2125; The Kicking Kangaroos – 1650
Feb: 4U2Rock – 2100; The Kicking Kangaroos – 1875; No One’s Darling-250; The Metalfolkies – 1000
Mar: 4U2Rock – 1975; The Kicking Kangaroos – 1575; No One’s Darling-1325; The Metalfolkies – 250
Apr: 4U2Rock – 1800; The Kicking Kangaroos – 1250; No One’s Darling-1600; The Metalfolkies – 500
May: 4U2Rock – 1700; The Kicking Kangaroos – 950; No One’s Darling-1725; The Metalfolkies – 650
Jun: 4U2Rock – 2025; The Kicking Kangaroos – 700; No One’s Darling-1825; The Metalfolkies – 900
Slide 13
Mathematics literacy – Example item
Question: CHARTS – Continued
In which month did the band No One’s Darling sell more CDs than the band The Kicking Kangaroos for the first time?
A No month
B March
C April
D May
SCORING
Correct
C April
Incorrect
Other responses.
Percentage of students answering correctly in 2012:
OECD average: 80%
U.S.: 77%
Slide 14
Mathematics literacy – Example item
Proficiency Level 5
Question: DRIP RATE
Intravenous drips are used to deliver fluids and drugs to patients.
(Picture of a nurse looking at an intravenous drip)
Nurses need to calculate the drip rate, D, in drops per minute for intravenous drips.
They use the formula D equals dv divided by 60n where
d is the drop factor measured in drops per milliliter (mL)
v is the volume in mL of the intravenous drip
n is the number of hours the intravenous drip is required to run.
3
Slide 15
Mathematics literacy – Example item
Question: DRIP RATE – Continued
Nurses also need to calculate the volume of the intravenous drip, v, from the drip rate, D.
An intravenous drip rate of 50 drops per minute has to be given to a patient for 3 hours. For this intravenous drip the
drop factor is 25 drops per milliliter.
What is the volume in mL of the intravenous drip?
Volume of the intravenous drip : _____________ mL
SCORING
Correct
360 or a correctly transposed and substituted solution.
 360
 (60 x 3 x 50) 25 [Correct transposition and substitution.]
Incorrect
Other responses.
Percentage of students answering correctly in PISA 2012:
OECD average: 26%
U.S.: 30%
Slide 16
Of 34 OECD Countries, U.S.A. Ranks 27th in Mathematics Literacy
2012 PISA- Mathematics Literacy
1st Korea 554; 2nd Japan 536; 3rd Switzerland 531; 4th Netherlands 523; 5th Estonia 521; 6th Finland 519; 7th Canada 518;
8th Poland 518; 9th Belgium 515; 10th Germany 514; 11th Austria 506; 12th Australia 504; 13th Ireland 501; 14th Slovenia
501; 15th Denmark 500; 16th New Zealand 500; 17th Czech Republic 499; 18th France 495; OECD average 494; 19th United
Kingdom 494; 20th Iceland 493; 21st Luxembourg 490; 22nd Norway 489; 23rd Portugal 487; 24th Italy 485; 25th Spain 484;
26th Slovak Republic 482; 27th United States 481; 28th Sweden 478; 29th Hungary 477; 30th Israel 466; 31st Greece 453;
32nd Turkey 448; 33rd Chile 423; 34th Mexico 413
Slide 17
U.S. 15-year-olds OECD average in mathematics literacy
Average higher than U.S. average
Shanghai-CHN 613; Singapore 573; Hong Kong-CHN 561; Taipei-CHN 560; Republic of Korea 554; Macao-CHN 538; Japan
536; Liechtenstein 535; Switzerland 531; Netherlands 523; Estonia 521; Finland 519; Canada 518; Poland 518; Belgium
515; Massachusetts-USA 514; Germany 514; Vietnam 511; Connecticut-USA 506; Austria 506; Australia 504; Ireland 501;
Slovenia 501; Denmark 500; New Zealand 500; Czech Republic 499; France 495; OECD average 494; United Kingdom 494;
Iceland 493; Latvia 491; Luxembourg 490
Average not measurably different from U.S. average
Norway 489; Portugal 487; Italy 485; Spain 484; Russian Federation 482; Slovak Republic 482; United States 481;
Lithuania 479; Sweden 478; Hungary 477
Average lower than U.S. average
4
Croatia 471; Florida-USA 467; Israel 466; Greece 453; Serbia 449; Turkey 448; Romania 445; Cyprus 440; Bulgaria 439;
United Arab Emirates 434; Kazakhstan 432; Thailand 427; Chile 423; Malaysia 421; Mexico 413; Montenegro 410;
Uruguay 409; Costa Rica 407; Albania 394; Brazil 391; Argentina 388; Tunisia 388; Jordan 386; Colombia 376; Qatar 376;
Indonesia 375; Peru 368
Massachusetts average higher than the U.S. and OECD averages, Connecticut average higher than the U.S. average but
not measurably different from the OECD average, and Florida average lower than the U.S. and OECD averages.
Slide 18
State results in mathematics literacy
Massachusetts: 9 education systems scored higher; 12 scored not measurably different; 44 scored lower
Connecticut: 12 education systems scored higher; 14 scored not measurably different; 39 scored lower
Florida: 36 education systems scored higher; 5 scored not measurably different; 24 scored lower
Slide 19
U.S. not measurably different from OECD average in two mathematics content subscales
Subscale
Quantity: U.S. average score 478; OECD average score 495. OECD average significantly higher than U.S. average
Uncertainty and data: U.S. average score 488; OECD average score 493. OECD average not measurably different from
U.S. average
Change and relationships: U.S. average score 488; OECD average score 493. OECD average not measurably different
from U.S. average
Space and shape: U.S. average score 463; OECD average score 490. OECD average significantly higher than U.S. average
Slide 20
In mathematics literacy, 9 percent of U.S. 15-year-old students scored at proficiency level 5 or above
Percentage at proficiency level 5 or above higher than U.S. percentage:
Shanghai-CHN, Below level 2: 4, Levels 5 and above: 55; Singapore, Below level 2: 8, Levels 5 and above: 40; Taipei-CHN,
Below level 2: 13, Levels 5 and above: 37; Hong Kong-CHN, Below level 2: 9, Levels 5 and above: 34; Republic of Korea,
Below level 2: 9, Levels 5 and above: 31; Liechtenstein, Below level 2: 14, Levels 5 and above: 25; Macao-CHN, Below
level 2: 11, Levels 5 and above: 24; Japan, Below level 2: 11, Levels 5 and above: 24; Switzerland, Below level 2: 12,
Levels 5 and above: 21; Belgium, Below level 2: 19, Levels 5 and above: 20; Netherlands, Below level 2: 15, Levels 5 and
above: 19; Massachusetts-USA, Below level 2: 18, Levels 5 and above: 19; Germany, Below level 2: 18, Levels 5 and
above: 17; Poland, Below level 2: 14, Levels 5 and above: 17; Connecticut-USA, Below level 2: 21, Levels 5 and above: 16;
Canada, Below level 2: 14, Levels 5 and above: 16; Finland, Below level 2: 12, Levels 5 and above: 15; New Zealand,
Below level 2: 23, Levels 5 and above: 15; Australia, Below level 2: 20, Levels 5 and above: 15; Estonia, Below level 2: 11,
Levels 5 and above: 15; Austria, Below level 2: 19, Levels 5 and above: 14; Slovenia, Below level 2: 20, Levels 5 and
above: 14; Vietnam, Below level 2: 14, Levels 5 and above: 13; France, Below level 2:22, Levels 5 and above: 13; Czech
Republic, Below level 2: 21, Levels 5 and above: 13; OECD average, Below level 2: 23, Levels 5 and above: 13; United
Kingdom, Below level 2: 22, Levels 5 and above: 12; Luxembourg, Below level 2: 24, Levels 5 and above: 11; Iceland,
Below level 2: 21, Levels 5 and above: 11; Ireland, Below level 2: 17, Levels 5 and above: 11.
5
Percentage not measurably different from U.S. percentage
Slovak Republic, Below level 2: 27, Levels 5 and above: 11; Portugal, Below level 2: 25, Levels 5 and above: 11; Denmark,
Below level 2: 17, Levels 5 and above: 10; Italy, Below level 2: 25, Levels 5 and above: 10; Norway, Below level 2: 22,
Levels 5 and above: 9; Israel, Below level 2: 34, Levels 5 and above: 9; Hungary, Below level 2: 28, Levels 5 and above: 9;
United States, Below level 2: 26, Levels 5 and above: 9; Lithuania, Below level 2: 26, Levels 5 and above: 8; Sweden,
Below level 2: 27, Levels 5 and above: 8; Spain, Below level 2: 24, Levels 5 and above: 8; Latvia, Below level 2: 20, Levels
5 and above: 8; Russian Federation, Below level 2: 24, Levels 5 and above: 8; Croatia, Below level 2: 30, Levels 5 and
above: 7.
Percentage at proficiency level 5 or above lower than U.S. percentage
Turkey, Below level 2: 42, Levels 5 and above: 6; Florida-USA, Below level 2: 30, Levels 5 and above: 6; Serbia, Below
level 2: 39, Levels 5 and above: 5; Bulgaria, Below level 2: 44, Levels 5 and above: 4; Greece, Below level 2: 36, Levels 5
and above: 4; Cyprus, Below level 2: 42, Levels 5 and above: 4; United Arab Emirates, Below level 2: 46, Levels 5 and
above: 3; Romania, Below level 2: 41, Levels 5 and above: 3; Thailand, Below level 2: 50, Levels 5 and above: 3; Qatar,
Below level 2: 70, Levels 5 and above: 2; Chile, Below level 2: 52, Levels 5 and above: 2; Uruguay, Below level 2: 56,
Levels 5 and above: 1; Malaysia, Below level 2: 52, Levels 5 and above: 1; Montenegro, Below level 2: 57, Levels 5 and
above: 1; Kazakhstan, Below level 2: 45, Levels 5 and above: 1; Albania, Below level 2: 61, Levels 5 and above: 1; Tunisia,
Below level 2: 68, Levels 5 and above: 1; Brazil, Below level 2: 67, Levels 5 and above: 1; Mexico, Below level 2: 55, Levels
5 and above: 1; Peru, Below level 2: 75, Levels 5 and above: 1; Costa Rica, Below level 2: 60, Levels 5 and above: 1;
Jordan, Below level 2: 69, Levels 5 and above: 0; Colombia, Below level 2: 74, Levels 5 and above: 0; Indonesia, Below
level 2: 76, Levels 5 and above: 0; Argentina, Below level 2: 66, Levels 5 and above: 0.
Massachusetts and Connecticut had higher percentages than the U.S and OECD; Florida’s percentage was lower than
those of the U.S. and OECD.
Slide 21
No measurable change since 2009 in average mathematics literacy scores in more than half of PISA education systems,
including U.S.
Of the 62 education systems in PISA 2009 and 2012:
In 33 (including the U.S.) no measurable change in average scores
In 18 average scores increased. Russian Federation was below U.S. and OECD averages in 2003; not measurably different
in 2012. Poland was lower than OECD and not measurably different from U.S. averages in 2003; higher than both in
2012.
In 11 average scores declined. Finland declined between 2006-2009 and again between 2009-2012; scored 548 in 2006
and 519 in 2012. Norway was above U.S. average in 2003; not measurably different from U.S. average and below OECD
average in 2012
Slide 22
Science Literacy
6
Slide 23
PISA science literacy proficiency levels
Level 1: Present scientific explanations that are obvious and follow explicitly from given evidence.
Level 2: Use direct reasoning and make literal interpretations of the results of scientific inquiry of technological problem
solving.
Level 3: Identify clearly described scientific issues in a range of contexts. Select facts and knowledge to explain
phenomena and apply simple models or inquiry strategies.
Level 4: Select and integrate explanations from different disciplines of science or technology and link those explanations
directly to aspects of life situations.
Level 5: Use well-developed inquiry abilities, link knowledge appropriately, and bring critical insights to situations.
Construct explanations based on evidence and arguments based on critical analysis.
Level 6: Link different information sources and explanations and use evidence from those sources to justify decisions.
Demonstrate advanced scientific thinking and reasoning, and use scientific understanding in support of solutions to
unfamiliar scientific situations.
Slide 24
Science literacy – Example item
Proficiency Level 2
Question: MARY MONTAGU
Read the following newspaper article and answer the questions that follow:
The History of Vaccination
Mary Mantagu was a beautiful woman. She survived an attack of smallpox in 1715 but was left covered with scars.
While living in Turkey in 1717, she observed a method called inoculation that was commonly used there. This treatment
involved scratching a weak type of smallpox virus onto the skin of healthy young people who then became sick, but in
most cases only with a mild form of the disease.
Mary Montagu was so convinced of the safety of these inoculations that she allowed her son and daughter to be
inoculated.
In 1796, Edward Jenner used inoculations of a related disease, cowpox, to produce antibodies against smallpox.
Compared with the inoculation of smallpox, this treatment had less side effects and the treated person could not infect
others. The treatment became known as vaccination.
Slide 25
Science literacy – Example item
Question: MARY MONTAGU – continued
What kinds of diseases can people be vaccinated against?
A Inherited diseases like hemophilia
B Diseases that are caused by viruses, like polio
C Diseases from the malfunctioning of the body, like diabetes
D Any sort of disease that has no cure
SCORING
Correct
Answer B. Diseases that are caused by viruses, like polio.
7
Incorrect
Other Responses
Percentage of students answering correctly in 2006:
OECD average: 75%
U.S.: 73%
Slide 26
Science literacy – Example item
Proficiency Level 6
Question: GREENHOUSE
Read the texts and answer the questions that follow.
The Greenhouse Effect: Fact or Fiction?
Living things need energy to survive. The energy that sustains life on Earth comes from the Sun, which radiates energy
into space because it is so hot. A tiny proportion of this energy reaches the Earth.
The Earth’s atmosphere acts like a protective blanket over the surface of our planet, preventing the variations in
temperature that would exist in an airless world.
Most of the radiated energy coming from the Sun passes through the Earth’s atmosphere. The Earth absorbs some of
this energy, and some is reflected back from the Earth’s surface. Part of this reflected energy is absorbed by the
atmosphere.
As a result of this, the average temperature above the Earth’s surface is higher than it would be if there was no
atmosphere. The Earth’s atmosphere has the same effect as a greenhouse, hence the term greenhouse effect.
The greenhouse effect is said to have become more pronounced during the twentieth century.
It is a fact that the average temperature of the Earth’s atmosphere has increased. In newspapers and periodicals the
increased carbon dioxide emission is often stated as the main source of the temperature rise in the twentieth century.
Slide 27
Science literacy – Example item
Question: GREENHOUSE – Continued
Andre persists in his conclusion that the average temperature rise of the Earth’s atmosphere is caused by the increase in
the carbon dioxide emission. But Jeanne thinks that his conclusion is premature. She says: “Before accepting this
conclusion you must be sure that other factors that could influence the greenhouse effect are constant”.
Name one of the factors that Jeanne means.
SCORING
Correct
Answers that give a factor referring to the energy/radiation coming from the Sun.
Answers that give a factor referring to a natural component or a potential pollutant.
Incorrect
Answers that refer to a cause that influences the carbon dioxide concentration.
Answers that refer to a non-specific factor.
Other incorrect factors or other responses.
Percentage of students answering correctly in 2006:
8
OECD average: 19%
U.S.: 18%
Slide 28
Of 34 OECD Countries, U.S.A. Ranks 20th in Science
2012 PISA- Science
1st Japan 547; 2nd Finland 545; 3rd Estonia 541; 4th Korea 538; 5th Poland 526; 6th Canada 525; 7th Germany 524; 8th
Netherlands 522; 9th Ireland 522; 10th Australia 521; 11th New Zealand 516; 12th Switzerland 515; 13th Slovenia 514; 14th
United Kingdom 514; 15th Czech Republic 508; 16th Austria 506; 17th Belgium 505; OECD average 501; 18th France 499;
19th Denmark 498; 20th United States 497; 21st Spain 496; 22nd Norway 495; 23rd Hungary 494; 24th Italy 494; 25th
Luxembourg 491; 26th Portugal 489; 27th Sweden 485; 28th Iceland 478; 29th Slovak Republic 471; 30th Israel 470; 31st
Greece 467; 32nd Turkey 463; 33rd Chile 445; 34th Mexico 415
Slide 29
U.S. 15-year-old students not measurably different from OECD average in science literacy
Average higher than U.S. average
Shanghai-CHN 580; Hong Kong-CHN 555; Singapore 551; Japan 547; Finland 545; Estonia 541; Republic of Korea 538;
Vietnam 528; Massachusetts-USA 527; Poland 526; Canada 525; Liechtenstein 525; Germany 524; Taipei-CHN 523;
Netherlands 522; Ireland 522; Australia 521; Connecticut-USA 521; Macao-CHN 521; New Zealand 516; Switzerland 515;
Slovenia 514; United Kingdom 514; Czech Republic 508.
Average not measurably different from U.S. average
Austria 506; Belgium 505; Latvia 502; OECD average 501; France 499; Denmark 498; United States 497; Spain 496;
Lithuania 496; Norway 495; Hungary 494; Italy 494; Croatia 491; Luxembourg 491; Portugal 489; Florida-USA 485.
Average lower than U.S. average
Russian Federation 486; Sweden 485; Iceland 478; Slovak Republic 471; Israel 470; Greece 467; Turkey 463; United Arab
Emirates 448; Bulgaria 446; Chile 445; Serbia 445; Thailand 444; Romania 439; Cyprus 438; Costa Rica 429; Kazakhstan
425; Malaysia 420; Uruguay 416; Mexico 415; Montenegro 410; Jordan 409; Argentina 406; Brazil 405; Colombia 399;
Tunisia 398; Albania 397; Qatar 384; Indonesia 382; Peru 373.
Massachusetts and Connecticut had averages significantly higher than the averages of the United States and the OECD.
Florida’s average was not significantly different from the United States but lower than the OECD average.
Slide 30
State results in science literacy
Massachusetts: 6 education systems scored higher; 14 scored not measurably different; 45 scored lower
Connecticut: 7 education systems scored higher; 15 scored not measurably different; 43 scored lower
Florida: 26 education systems scored higher; 15 scored not measurably different; 24 scored lower
Slide 31
In science literacy, 7 percent of U.S. 15-year-old students scored at proficiency level 5 or above
Percentage at proficiency level 5 or above higher than U.S. percentage
Shanghai-CHN, Below level 2: 3, Levels 5 and above: 27; Singapore, Below level 2: 10, Levels 5 and above: 23; Japan,
Below level 2: 8, Levels 5 and above: 18; Finland, Below level 2: 8, Levels 5 and above: 17; Hong Kong-CHN, Below level
2: 6, Levels 5 and above: 17; Massachusetts-USA, Below level 2: 11, Levels 5 and above: 14; Australia, Below level 2: 14,
Levels 5 and above: 14; New Zealand, Below level 2: 16, Levels 5 and above: 13; Connecticut-USA, Below level 2: 13,
9
Levels 5 and above: 13; Estonia, Below level 2: 5, Levels 5 and above: 13; Germany, Below level 2: 12, Levels 5 and
above: 12; Netherlands, Below level 2: 13, Levels 5 and above: 12; Republic of Korea, Below level 2: 7, Levels 5 and
above: 12; Canada, Below level 2: 10, Levels 5 and above: 11; United Kingdom, Below level 2: 15, Levels 5 and above: 11;
Poland, Below level 2: 9, Levels 5 and above: 11; Ireland, Below level 2: 11, Levels 5 and above: 11; Slovenia, Below level
2: 13, Levels 5 and above: 10.
Percentage not measurably different from U.S. percentage
Liechtenstein, Below level 2: 10, Levels 5 and above: 10; Switzerland, Below level 2: 13, Levels 5 and above: 9; Belgium,
Below level 2: 18, Levels 5 and above: 9; OECD average, Below level 2: 18, Levels 5 and above: 8; Taipei-CHN, Below level
2: 10, Levels 5 and above: 8; Luxembourg, Below level 2: 22, Levels 5 and above: 8; Vietnam, Below level 2: 7, Levels 5
and above: 8; France, Below level 2: 19, Levels 5 and above: 8; Austria, Below level 2: 16, Levels 5 and above: 8; Czech
Republic, Below level 2: 14, Levels 5 and above: 8; Norway, Below level 2: 20, Levels 5 and above: 8; United States,
Below level 2: 18, Levels 5 and above: 7; Denmark, Below level 2: 17, Levels 5 and above: 7; Macao-CHN, Below level 2:
9, Levels 5 and above: 7; Sweden, Below level 2: 22, Levels 5 and above: 6; Italy, Below level 2: 19, Levels 5 and above: 6;
Hungary, Below level 2: 18, Levels 5 and above: 6; Israel, Below level 2: 29, Levels 5 and above: 6; Florida-USA, Below
level 2: 21, Levels 5 and above: 5.
Percentage at proficiency level 5 or above lower than U.S. percentage
Iceland, Below level 2: 24, Levels 5 and above: 5; Lithuania, Below level 2: 16, Levels 5 and above: 5; Slovak Republic,
Below level 2: 27, Levels 5 and above: 5; Spain, Below level 2: 16, Levels 5 and above: 5; Croatia, Below level 2: 17,
Levels 5 and above: 5; Portugal, Below level 2: 19, Levels 5 and above: 5; Latvia, Below level 2: 12, Levels 5 and above:
4; Russian Federation, Below level 2: 19, Levels 5 and above: 4; Bulgaria, Below level 2: 37, Levels 5 and above: 3; United
Arab Emirates, Below level 2: 35, Levels 5 and above: 3; Greece, Below level 2: 26, Levels 5 and above: 2; Cyprus, Below
level 2: 38, Levels 5 and above: 2; Turkey, Below level 2: 26, Levels 5 and above: 2; Serbia, Below level 2: 35, Levels 5 and
above: 2; Qatar, Below level 2: 63, Levels 5 and above: 1; Uruguay, Below level 2: 47, Levels 5 and above: 1; Chile, Below
level 2: 34, Levels 5 and above: 1; Thailand, Below level 2: 34, Levels 5 and above: 1; Romania, Below level 2: 37, Levels
5 and above: 1; Albania, Below level 2: 53, Levels 5 and above: 0; Montenegro, Below level 2: 51, Levels 5 and above: 0;
Malaysia, Below level 2: 46, Levels 5 and above: 0; Brazil, Below level 2: 54, Levels 5 and above: 0; Jordan, Below level 2:
50, Levels 5 and above: 0; Argentina, Below level 2: 51, Levels 5 and above: 0; Costa Rica, Below level 2: 39, Levels 5 and
above: 0; Kazakhstan, Below level 2: 42, Levels 5 and above: 0; Mexico, Below level 2: 47, Levels 5 and above: 0;
Colombia, Below level 2: 56, Levels 5 and above: 0; Tunisia, Below level 2: 55, Levels 5 and above: 0; Indonesia, Below
level 2: 67, Levels 5 and above: 0; Peru, Below level 2: 68, Levels 5 and above: 0.
Massachusetts and Connecticut had significantly higher percentages than the U.S. and OECD; Florida’s percentage was
not significantly different from the percentage of the U.S. but lower than the OECD’s.
Slide 32
No measurable change since 2009 in average science literacy scores in more than half of PISA education systems,
including the U.S.
Of the 62 education systems that participated in PISA 2009 and 2012:
In 43 (including the U.S.) there was no measurable change in average scores.
In 13 average scores increased. Poland was not measurably different from U.S. and OECD averages in 2006; above U.S.
and OECD averages in 2012.
In 6 average scores declined. Sweden was higher than U.S. and OECD averages in 2006; below U.S. and OECD averages in
2012.
Slide 33
Reading Literacy
10
11
Slide 34
PISA reading literacy proficiency levels
Level 1b: Locate a single piece of explicitly stated information in a prominent position in a short, simple text.
Level 1a: Recognize the main theme or author’s purpose in a text about a familiar topic.
Level 2: Locate one or more pieces of information, which may need to be inferred and may need to meet several
conditions.
Level 3: Integrate several parts of a text in order to identify a main idea, understand a relationship, or construe the
meaning of a word or phrase.
Level 4: Interpret the meaning of nuances of language in a section of text by taking into account the text as a whole.
Level 5: Locate and organize several pieces of deeply embedded information, inferring which information in the text is
relevant.
Level 6: Make multiple inferences, comparisons, and contrasts that are both detailed and precise. Deal with unfamiliar
ideas and generate abstract categories for interpretations.
Slide 35
Reading literacy – Example item
Proficiency level 3
Question: TELECOMMUTING
The way of the future
Just imagine how wonderful it would be to “telecommute”1 to work on the electronic highway, with all your work done
on a computer or by phone! No longer would you have to jam your body into crowded buses or trains or waste hours
and hours traveling to and from work. You could work wherever you want to – just think of all the job opportunities this
would open up! Molly
Disaster in the making
Cutting down on commuting hours and reducing the energy consumption involved is obviously a good idea. But such a
goal should be accomplished by improving public transportation or by ensuring that workplaces are located near where
people live. The ambitious idea that telecommuting should be part of everyone’s way of life will only lead to people
becoming more and more self-absorbed. Do we really want our sense of being part of a community to deteriorate even
further? Richard
1
“Telecommuting” is a term coined by Jack Niles in the early 1970s to describe a situation in which workers work on a
computer away from a central office (for example, at home) and transmit data and documents to the central office via
telephone lines.
Slide 36
Reading literacy – Example item
Question: TELECOMMUTING-Continued
What is the relationship between “The way of the future” and “Disaster in the making”?
A
B
C
D
They use different arguments to reach the same general conclusion.
They are written in the same style but they are about completely different topics.
They express the same general point of view, but arrive at different conclusions.
They express opposing points of view on the same topic.
SCORING
Correct
12
Answer D. They express opposing points of view on the same topic.
Incorrect
Other responses.
Percentage of students answering correctly in 2009:
OECD average: 52%
U.S.: 55%
Slide 37
Reading literacy – Example item
Proficiency level 6
Question: THE PLAY’S THE THING
Takes place in a castle by the beach in Italy.
First Act
Ornate guest room in a very nice beachside castle. Doors on the right and left. Sitting room set in the middle of the
stage: couch, table, and two armchairs. Large windows at the back. Starry night. It is dark on the stage. When the
curtain goes up we hear men conversing loudly behind the door on the left. The door opens and three tuxedoed
gentlemen enter. One turns the light on immediately. They walk to the center in silence and stand around the table.
They sit down together, Gal in the armchair to the left, Turai in the one on the right, Adam on the couch in the middle.
Very long, almost awkward silence. Comfortable stretches. Silence. Then:
GAL: Why are you so deep in thought?
TURAI: I’m thinking about how difficult it is to begin a play. To introduce all the principal characters in the beginning,
when it all starts.
ADAM: I suppose it must be hard.
TURAI: It is – devilishly hard. The play starts. The audience goes quiet. The actors enter the stage and the torment
begins. It’s an eternity, sometimes as much as a quarter of an hour before the audience finds out who’s who and what
they are all up to.
GAL: Quite a peculiar brain you’ve got. Can’t you forget your profession for a single minute?
TURAI: That cannot be done.
GAL: Not half an hour passes without you discussing theatre, actors, plays. There are other things in this world.
TURAI: There aren’t. I am a dramatist. This is my curse.
GAL: You shouldn’t become such a slave to your profession.
TURAI: If you do not master it, you are its slave. There is no middle ground. Trust me, it’s no joke starting a play well. It
is one of the toughest problems of stage mechanics. Introducing your characters promptly. Let’s look at this scene here,
the three of us. Three gentlemen in tuxedoes. Say they enter not this room in this lordly castle, but rather a stage, just
when a play begins. They would have to chat about a whole lot of uninteresting topics until it came out who we are.
Wouldn’t it be much easier to start all this by standing up and introducing ourselves? Stands up. Good evening. The
three of us are guests in this castle. We have just arrived from the dining room where we had an excellent dinner and
13
drank two bottles of champagne. My name is Sandor Turai, I’m a playwright, I’ve been writing plays for thirty years,
that’s my profession. Full stop. Your turn.
GAL: Stands up. My name is Gal, I’m also a playwright. I write plays as well, all of them in the company of this
gentleman here. We are a famous playwright duo. All playbills of good comedies and operettas read: written by Gal
and Turai. Naturally, this is my profession as well.
GAL and TURAI: Together. And this young man . . .
ADAM: Stands up. This young man is, if you allow me, Albert Adam, twenty-five years old, composer. I wrote the music
for these kind gentlemen for their latest operetta. This is my first work for the stage. These two elderly angels have
discovered me and now, with their help, I’d like to become famous. They got me invited to this castle. They got my
dress-coat and tuxedo made. In other words, I am poor and unknown, for now. Other than that I’m an orphan and my
grandmother raised me. My grandmother has passed away. I am all alone is this world. I have no name, I have no
money.
TURAI: But you are young.
GAL: And gifted.
ADAM: And I am in love with the soloist.
TURAI: You shouldn’t have added that. Everyone in the audience would figure that out anyway.
They all sit down.
Slide 38
Reading literacy – Example item
Question: THE PLAY’S THE THING - Continued
What were the characters in the play doing just before the curtain went up?
SCORING
Correct
Answer which refers to dinner or drinking champagne. May paraphrase or quote the text directly.
They have just had dinner and champagne
“We have just arrived from the dining room where we had an excellent dinner.” [direct quotation]
“An excellent dinner and drank two bottles of champagne.” [direct quotation]
Incorrect
Answers which give an insufficient or vague response, show inaccurate comprehension of the material, or are
implausible or irrelevant.
Percentage of students answering correctly in PISA 2009:
OECD average: 13%
U.S.: 13%
14
Slide 39
Of 34 OECD Countries, U.S.A. Ranks 17th in Reading
2012 PISA- Reading
1st Japan 538; 2nd Korea 536; 3rd Finland 524; 4th Ireland 523; 5th Canada 523; 6th Poland 518; 7th Estonia 516; 8th New
Zealand 512; 9th Australia 512; 10th Netherlands 511; 11th Belgium 509; 12th Switzerland 509; 13th Germany 508; 14th
France 505; 15th Norway 504; 16th United Kingdom 499; 17th United States 498; OECD average 496; 18th Denmark 496;
19th Czech Republic 493; 20th Italy 490; 21st Austria 490; 22nd Hungary 488; 25th Portugal 488; 26th Israel 486; 23rd Spain
488; 24th Luxembourg 488; 27th Sweden 483; 28th Iceland 483; 29th Slovenia 481; 30th Greece 477; 31st Turkey 475; 32nd
Slovak Republic 463; 33rd Chile 441; 34th Mexico 424
Slide 40
U.S. 15-year-olds not measurably different from OECD average in reading literacy
Average higher than U.S. average
Shanghai-CHN 570; Hong Kong-CHN 545; Singapore 542; Japan 538; Republic of Korea 536; Massachusetts-USA 527;
Finland 524; Ireland 523; Taipei-CHN 523; Canada 523; Connecticut-USA 521; Poland 518; Estonia 516; Liechtenstein
516; New Zealand 512; Australia 512; Netherlands 511; Switzerland 509; Macao-CHN 509; Belgium 509; Germany 508.
Average not measurably different from U.S. average
Vietnam 508; France 505; Norway 504; United Kingdom 499; United States 498; OECD average 496; Denmark 496; Czech
Republic 493; Florida-USA 492; Italy 490; Austria 490; Hungary 488; Portugal 488; Israel 486.
Average lower than U.S. average
Latvia 489; Spain 488; Luxembourg 488; Croatia 485; Sweden 483; Iceland 483; Slovenia 481; Lithuania 477; Greece 477;
Turkey 475; Russian Federation 475; Slovak Republic 463; Cyprus 449; Serbia 446; United Arab Emirates 442; Chile 441;
Thailand 441; Costa Rica 441; Romania 438; Bulgaria 436; Mexico 424; Montenegro 422; Uruguay 411; Brazil 410; Tunisia
404; Colombia 403; Jordan 399; Malaysia 398; Indonesia 396; Argentina 396; Albania 394; Kazakhstan 393; Qatar 388;
Peru 384.
Massachusetts and Connecticut had average scores significantly higher than the United States and the OECD averages.
Florida’s average was not measurably different from the averages of United States and the OECD.
Slide 41
State results in reading literacy
Massachusetts: 3 education systems scored higher; 9 scored not measurably different; 53 scored lower
Connecticut: 4 education systems scored higher; 16 scored not measurably different; 45 scored lower
Florida: 21 education systems scored higher; 17 scored not measurably different; 27 scored lower
Slide 42
In reading literacy, 8 percent of U.S. 15-year-old students scored at proficiency level 5 or above, not measurably
different from the OECD average
Percentage at proficiency level 5 or above higher than U.S. percentage
Shanghai-CHN, Below level 2: 3, Levels 5 and above: 25; Singapore, Below level 2: 10, Levels 5 and above: 21; Japan,
Below level 2: 10, Levels 5 and above: 18; Hong Kong-CHN, Below level 2: 7, Levels 5 and above: 17; Massachusetts,
Below level 2: 11, Levels 5 and above: 16; Connecticut, Below level 2: 13, Levels 5 and above: 15; Republic of Korea,
15
Below level 2: 8, Levels 5 and above: 14; New Zealand, Below level 2: 16, Levels 5 and above: 14; Finland, Below level 2:
11, Levels 5 and above: 13; France, Below level 2: 19, Levels 5 and above: 13; Canada, Below level 2: 11, Levels 5 and
above: 13; Taipei-CHN, Below level 2: 11, Levels 5 and above: 12; Belgium, Below level 2: 16, Levels 5 and above: 12
Australia, Below level 2: 14, Levels 5 and above: 12; Ireland, Below level 2: 10, Levels 5 and above: 11; Norway, Below
level 2: 16, Levels 5 and above: 10.
Percentage not measurably different from U.S. percentage
Liechtenstein, Below level 2: 12, Levels 5 and above: 11; Poland, Below level 2: 11, Levels 5 and above: 10; Netherlands,
Below level 2: 14, Levels 5 and above: 10; Israel, Below level 2: 24, Levels 5 and above: 10; Switzerland, Below level 2:
14, Levels 5 and above: 9; Germany, Below level 2: 14, Levels 5 and above: 9; Luxembourg, Below level 2: 22, Levels 5
and above: 9; United Kingdom, Below level 2: 17, Levels 5 and above: 9; OECD average, Below level 2: 18, Levels 5 and
above: 8; Estonia, Below level 2: 9, Levels 5 and above: 8; United States, Below level 2: 17, Levels 5 and above: 8;
Sweden, Below level 2: 23, Levels 5 and above: 8; Macao-CHN, Below level 2: 11, Levels 5 and above: 7; Italy, Below level
2: 20, Levels 5 and above: 7.
Percentage at proficiency level 5 or above lower than U.S. percentage
Czech Republic, Below level 2: 17, Levels 5 and above: 6; Iceland, Below level 2: 21, Levels 5 and above: 6; Portugal,
Below level 2: 19, Levels 5 and above: 6; Hungary, Below level 2: 20, Levels 5 and above: 6; Spain, Below level 2: 18,
Levels 5 and above: 6; Austria, Below level 2: 19, Levels 5 and above: 6; Florida-USA, Below level 2: 17, Levels 5 and
above: 6; Denmark, Below level 2: 15, Levels 5 and above: 5; Greece, Below level 2: 23, Levels 5 and above: 5; Slovenia,
Below level 2: 21, Levels 5 and above: 5; Russian Federation, Below level 2: 22, Levels 5 and above: 5; Vietnam, Below
level 2: 9, Levels 5 and above: 5; Croatia, Below level 2: 19, Levels 5 and above: 4; Slovak Republic, Below level 2: 28,
Levels 5 and above: 4; Turkey, Below level 2: 22, Levels 5 and above: 4; Bulgaria, Below level 2: 39, Levels 5 and above: 4;
Latvia, Below level 2: 17, Levels 5 and above: 4; Cyprus, Below level 2: 33, Levels 5 and above: 4; Lithuania, Below level 2:
21, Levels 5 and above: 3; Serbia, Below level 2: 33, Levels 5 and above: 2; United Arab Emirates, Below level 2: 36,
Levels 5 and above: 2; Qatar, Below level 2: 57, Levels 5 and above: 2; Romania, Below level 2: 37, Levels 5 and above: 2;
Albania, Below level 2: 52, Levels 5 and above: 1; Montenegro, Below level 2: 43, Levels 5 and above: 1; Uruguay, Below
level 2: 47, Levels 5 and above: 1; Thailand, Below level 2: 33, Levels 5 and above: 1; Chile, Below level 2: 33, Levels 5
and above: 1; Costa Rica, Below level 2: 32, Levels 5 and above: 1; Argentina, Below level 2: 54, Levels 5 and above: 1;
Brazil, Below level 2: 49, Levels 5 and above: 1; Peru, Below level 2: 60, Levels 5 and above: 0; Mexico, Below level 2: 41,
Levels 5 and above: 0; Colombia, Below level 2: 51, Levels 5 and above: 0; Tunisia, Below level 2: 49, Levels 5 and above:
0; Jordan, Below level 2: 51, Levels 5 and above: 0; Malaysia, Below level 2: 53, Levels 5 and above: 0; Indonesia, Below
level 2: 55, Levels 5 and above: 0; Kazakhstan, Below level 2: 57, Levels 5 and above: 0.
Massachusetts and Connecticut had higher percentages than the U.S. and OECD; Florida’s percentage was lower than
the U.S. and OECD.
Slide 43
No measurable change since 2009 in average reading literacy scores in more than half of PISA education systems,
including the U.S.
Of the 62 education systems that participated in PISA 2009 and 2012:
In 34 (including the U.S.) no measurable change in average scores
In 21 average scores increased. Germany was lower than U.S. and OECD averages in 2000; higher than U.S. and OECD
averages in 2012
In 7 average scores declined. Sweden was higher than OECD average and not different from U.S. in 2000; lower than
both U.S. and OECD averages in 2012.
16
Slide 44
Among OECD Countries, U.S. has the 4th Largest Achievement Gap Between High-SES and Low-SES Students
2006 PISA- Science Literacy
1st France: High 559, Low 448, Gap 111; 2nd Luxembourg: High 540, Low 429, Gap 111; 3rd Belgium: High 564, Low 464,
Gap 100; 4th United States: High 541, Low 445, Gap 96; 5th United Kingdom: High 568, Low 472, Gap 96; 6th Switzerland:
High 559, Low, 463, Gap 96; 7th Czech Republic: High 564, Low 469, Gap 95; 8th Netherlands: High 573, Low 478, Gap 95;
9th Germany: High 569, Low 474, Gap 95; 10th Slovak Republic: High 538, Low 446, Gap 92; 11th New Zealand: High 578,
Low 487, Gap 91; 12th Austria: High 554, Low 466, Gap 88; 13th Greece: High 518, Low 436, Gap 82; 14th Hungary: High
552, Low 469, Gap 83; 15th Portugal: High 521, Low 440, Gap 81; 16th Sweden: High 547, Low 467, Gap 80; OECD
average: High 544, Low 464, Gap 80; 17th Australia: High 569, Low 490, Gap 79; 18th Spain: High 530, Low 455, Gap 75;
19th Denmark: High 537, Low 463, Gap 74; 20th Ireland: High 548, Low 476, Gap 72; 21st Norway: High 529, Low 457, Gap
72; 22nd Mexico: High 448, Low 377, Gap 71; 23rd Italy: High 512, Low 442, Gap 70; 24th Poland: High 539, Low 469, Gap
70; 25th Canada: High 569, Low 502, Gap 67; 26th Turkey: High 461, Low 404, Gap 57; 27th Iceland: High 525, Low 468,
Gap 57; 28th Japan: High 559, Low 506, Gap 53; 29th Finland: High 596, Low 545, Gap 51; 30th Korea: High 545, Low 506,
Gap 39
Slide 45
Among OECD Countries, U.S. has the 5th Largest Achievement Gap Between High-SES and Low-SES Students
2009 PISA-Reading Literacy
1st Hungary: High 553, Low 435, Gap 118; 2nd Belgium: High 567, Low 452, Gap 115; 3rd Luxembourg: High 526, Low 411,
Gap 115; 4th France: High 553, Low 443, Gap 110; 5th United States: High 544, Low 451, Gap 93; 6th Austria: High 525,
Low 421, Gap 104; 7th Germany: High 550, Low 445, Gap 105; 8th New Zealand: High 578, Low 475, Gap 103; 9th Israel:
High 526, Low 423, Gap 103; 10th Switzerland: High 550, Low 457, Gap 93; 11th United Kingdom: High 544, Low 451, Gap
93; 12th Turkey: High 514, Low 422, Gap 92; 13th Chile: High 501, Low 409, Gap 92; 14th Greece: High 528, Low 437, Gap
91; 15th Sweden: High 543, Low 452, Gap 91; 16th Australia: High 562, Low 471, Gap 91; OECD average: High 540, Low
451, Gap 89; 17th Slovenia: High 532, Low 444, Gap 88; 18th Poland: High 550, Low 461, Gap 89; 19th Portugal: High 537,
Low 451, Gap 86; 20th Slovak Republic: High 521, Low 435, Gap 86; 21st Ireland: High 539, Low 454, Gap 85; 22nd Italy:
High 526, Low 442, Gap 84; 23rd Czech Republic: High 521, Low 437, Gap 84; 24th Mexico: High 469, Low 386, Gap 83;
25th Spain: High 525, Low 443, Gap 82; 26th Denmark: High 536, Low 455, Gap 81; 27th Netherlands: High 553, Low 474,
Gap 79; 28th Japan: High 558, Low 483, Gap 75; 29th Korea: High 572, Low 303, Gap 69; 30th Norway: High 536, Low 468,
Gap 68; 31st Canada: High 562, Low 495, Gap 67; 32nd Finland: High 565, Low 504, Gap: 61; 33rd Iceland: High 530, Low
470, Gap 60; 34th Estonia: High 534, Low 476, Gap 58
Slide 46
The U.S. ranks 26th among 34 OECD Countries on the Percentage of Low-SES Students who are High-Performing
PISA 2012-Mathematics
1st Korea 12.7; 2nd Japan 11.3; 3rd Switzerland 9.9; 4th Estonia 9.5; 5th Netherlands 8.6; 6th Poland 8.4; 7th Canada 8.3; 8th
Finland 8.1; 9th Belgium 7.1; 10th Portugal 7.7; 11th Germany 7.5; 12th Turkey 7.2; OECD average 6.4; 13th Italy 6.4; 14th
Spain 6.4; 15th Ireland 6.3; 16th Australia 6.3; 17th Austria 6.1; 18th Luxembourg 6.1; 19th Czech Republic 5.9; 20th Slovenia
5.9; 21st United Kingdom 5.8; 22nd France 5.4; 23rd Norway 5.3; 24th Iceland 5.2; 25th New Zealand 5.3; 26th United States
5.2; 27th Denmark 4.9; 28th Sweden 4.3; 29th Hungary 4.1; 30th Slovak Republic 3.9; 31st Mexico 3.9; 32nd Greece 3.2; 33rd
Israel 3.1; 34th Chile 1.7
17
Slide 47
The U.S. Achievement Gap Between High-SES and Low-SES Students is Equivalent to Over Two Years of Schooling
2012 PISA-Mathematics
Slovak Republic: High 545, Low 416, Gap 129; France: High 561, Low 442, Gap 119; Hungary: High 539, Low 422, Gap
117; Israel: High 524, Low 409, Gap 115; New Zealand: High 559, Low 444, Gap 115; Luxembourg: High 546, Low 438,
Gap 108; Portugal: High 548, Low 441, Gap 107; Czech Republic: High 552, Low 450, Gap 102; Germany: High 569, Low
467, Gap 102; Chile: High 477, Low 378, Gap 99; Belgium: High 575, Low 460, Gap 115; Poland: High 571, Low 473, Gap
98; Austria: High 552, Low 458, Gap 94; Slovenia: High 552, Low 458, Gap 94; United States: High 532, Low 442, Gap 90;
Spain: High 533, Low 442, Gap 91; OECD average: High 542, Low 452, Gap 90; Greece: High 502, Low 413, Gap 89;
Switzerland: High 576, Low 488, Gap 88; Australia: High 550, Low 463, Gap 87; United Kingdom: High 545, Low 458, Gap
87; Turkey: High 498, Low 412, Gap 86; Denmark: High 545, Low 460, Gap 85; Ireland: High 545, Low 462, Gap 83;
Netherlands: High 565, Low 484, Gap 81; Korea: High 595, Low 516, Gap 79; Japan: High 575, Low 500, Gap 75; Italy:
High 522, Low 447, Gap 75; Sweden: High 518, Low 443, Gap 75; Canada: High 558, Low 486, Gap 72; Finland: High 555,
Low 488, Gap 67; Norway: High 522, Low 459, Gap 63; Estonia: High 559, Low 496, Gap 63; Iceland: High 526, Low 464,
Gap 62; Mexico: High 447, Low 385, Gap 62
18