© 2011 California County Superintendents Educational Services Association
4/13/2015
1
Benchmarking Definitions
“A process of comparing the quality of a specific
process or method to another that is widely considered
to be an industry standard or best practice.”
“Benchmarking is the practice of being humble enough
to admit that someone else has a better process and
wise enough to learn how to match or even
surpass them.”
American Productivity and Quality Center from Benchmarking for Success:
Ensuring U.S. Students Receive a World-Class Education
a report by the NGA, CCSSO and Achieve
© 2011 California County Superintendents Educational Services Association
4/13/2015
2
Goals of International Comparisons




Gain insight to improve educational
systems
Access global knowledge and best
practices
Necessary to compete in our global
economy
Compare educational performance
over time
© 2011 California County Superintendents Educational Services Association
4/13/2015
3
International Comparison:
Areas of Educational Benchmarking

Standards

Accountability

Educator workforce

Assessment
© 2011 California County Superintendents Educational Services Association
4/13/2015
4
Characteristics of Standards in the
best-performing nations

Focus: Deeper

Rigor: Complex and challenging

Coherence: Orderly progression

Fluency: Proficient and automatic
© 2011 California County Superintendents Educational Services Association
4/13/2015
5
© 2011 California County Superintendents Educational Services Association
4/13/2015
6
Mathematics
Benchmarking International
Standards
© 2011 California County Superintendents Educational Services Association
4/13/2015
7
Focus for Math Assessments
Same skills
assessed across
countries
International
data provided
since 1995
Focus on
international
economics &
employment
© 2011 California County Superintendents Educational Services Association
4/13/2015
8
International Assessments: Mathematics
Programme for International Student
Assessment (PISA)
Assessments:
• Focus on determining if students can use what they
have learned in the situations they are likely to encounter
in their daily life
• Have students analyze, reason, and communicate
effectively as they pose, formulate, solve and interpret
mathematical problems in a variety of situations
• Test how students apply skills to real-world problems
© 2011 California County Superintendents Educational Services Association
4/13/2015
9
PISA Sample Problem
1. What is the approximate distance from the starting line to the
beginning of the longest straight section of the track?
2. What is the lowest speed recorded during the second lap?
3. What can you say about the speed of the car between the 2.6 km
and 2.8 km marks?
© 2011 California County Superintendents Educational Services Association
4/13/2015
10
International Assessments: Mathematics
Trends in International Mathematics and
Science Study (TIMSS)
Assessments:
•Place emphasis on questions and tasks that offer better
insight into students’ analytical, problem solving and inquiry
skills and capabilities
•Test both content and cognitive domains
Content: Number, Geometry, Algebra, Data and
Chance, etc.
Cognitive: Knowing, Applying, Reasoning
© 2011 California County Superintendents Educational Services Association
4/13/2015
11
TIMSS Sample Problem


John and Cathy were told to divide a number by
100. By mistake John multiplied the number by
100 and obtained an answer of 450. Cathy
correctly divided the number by 100. What was her
answer?
If n is a negative integer, which of these is the
largest number?
a) 3 + n
b) 3 × n
c) 3 − n
d) 3 ÷ n
© 2011 California County Superintendents Educational Services Association
4/13/2015
12
International Benchmarking:
Composite Standards for Mathematics




A model to compare US standards with three
high-performing countries: Hong Kong,
Singapore and Korea.
Focus on learning progressions
Transform the Asian standards into an
organization of mathematics content
Create a set of composite standards by topic
and grade.
Informing Grades 1-6 Mathematics Standards Development:
What Can be Learned from High-performing Hong Kong, Korea and Singapore.
American Institutes for Research 2009
© 2011 California County Superintendents Educational Services Association
4/13/2015
13
Composite Standard Development



Identify the core mathematics topics taught
within each strand across countries
Identify each country’s grade-by-grade
sequence of mathematical competencies for
each core topic
Create composite standards for each core topic
by drawing from the learning progressions in
the standards in each of the three countries
© 2011 California County Superintendents Educational Services Association
4/13/2015
14
Benchmarking International
Standards
© 2011 California County Superintendents Educational Services Association
4/13/2015
15
Focus on Reading Literacy Assessments
Inclusion of
reading,
mathematical &
science literacy
International
data provided
since 2001
Emphasis on reading
comprehension in
country’s native
language
© 2011 California County Superintendents Educational Services Association
4/13/2015
16
Introduction: Reading Literacy
Major international assessments:
• The Progress in Reading Literacy Study (PIRLS)
• Programme for International Student Assessment
(PISA)
© 2011 California County Superintendents Educational Services Association
4/13/2015
17
The Progress in Reading
Literacy Study: PIRLS
© 2011 California County Superintendents Educational Services Association
4/13/2015
18



Measure trends in reading literacy
achievement.
Improve the teaching of reading and the
acquisition of reading skills around the
world.
Assess reading achievement for students in
their fourth year of school.
© 2011 California County Superintendents Educational Services Association
4/13/2015
19
PIRLS defines reading literacy as the
ability to understand and use those
written language forms required by
society and/or valued by the
individual. Young readers can
construct meaning from a variety of
texts. They read to learn, to participate
in communities of readers in school
and everyday life, and for enjoyment.
© 2011 California County Superintendents Educational Services Association
4/13/2015
20

The PIRLS assessment focuses on the two
overarching purposes for reading :
• Reading for literary experience (50%)
• Reading to acquire and use information (50%)
© 2011 California County Superintendents Educational Services Association
4/13/2015
21
PIRLS: Comprehension Processes




Focus on and retrieve explicitly stated
information (20%)
Make straightforward inferences (30%)
Interpret and integrate ideas and
information and textual elements (30%)
Examine and evaluate content, language,
and textual elements (20%)
© 2011 California County Superintendents Educational Services Association
4/13/2015
22
Programme for International
Student Assessment: PISA
© 2011 California County Superintendents Educational Services Association
4/13/2015
23
PISA: Purpose


Measure how well students are
prepared to meet the challenges
they may encounter in future life.
Tested Age-15
© 2011 California County Superintendents Educational Services Association
4/13/2015
24
PISA: Reading Literacy Definition
Reading literacy is understanding,
using, reflecting on and engaging with
written texts, in order to achieve one’s
goals, to develop one’s knowledge and
potential, and to participate in society.
© 2011 California County Superintendents Educational Services Association
4/13/2015
25
PISA Skills

Access and retrieve

Integrate and interpret

Reflect and evaluate
© 2011 California County Superintendents Educational Services Association
4/13/2015
26
References

Benchmarking for Success: Ensuring U.S. Students Receive a World-Class Education. National
Governors Association, Council of Chief State School Officers, and Achieve, Inc., 2008
(http://www.achieve.org/benchmarkingforsuccess)

Comparative Indicators of Education in the United States and Other G-8 Counties: 2009
(http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2009039)

IEA’s Progress in International Reading Literacy Study in Primary School in 40 Countries

Overview of IEA’s PIRLS Assessment 2011

PISA 2009 Assessment Framework: Key competencies in reading, mathematics and science
(http://timss.bc.edu/pirls2006/intl_rpt.html)
http://www.timss.bc.edu/pirls2011/download PIRLS2011_Framework.pdf)
(http://www.oecd.org/dataoecd/11/40/44455820.pdf)

Informing Grades 1 – 6 Mathematics Standards Development: What Can Be Learned From
High-Performing Hong Kong, Korea , and Singapore?
(http://www.air.org/focus-area/education/index.cfm?fa=viewContent&content_id=670)


The Second Derivative: International Benchmarks in Mathematics for U.S. States and School
Districts (http://hub.mspnet.org/index.cfm/20749)
U. S. Performance Across International Assessments of Student Achievement: Special
Supplement to The Condition of Education 2009 National Center for Education Statistics
(http://nces.ed.gov/pubsearch/pubsinfo.asp?pubid=2009083)

Foundations for Success: The Final Report of the National Mathematics Advisory Panel. US
Department of Education, 2008 (http://www2.ed.gov/about/bdscomm/list/mathpanel/report/final-report.pdf )
© 2011 California County Superintendents Educational Services Association
4/13/2015
27