NAEP, TIMSS, PISA and PIRLS
September 8, 2015
DAC Annual Meeting
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
Update
There is a new NAEP Assistant in town!
Alexandra Fahnlander
• Alex is from Fort Myers
• Graduated from FSU in May
with a degree in English,
focusing on editing, writing,
and media
• Worked in the arts and at the
FSU radio station during college
• Volunteers with Florida’s YMCA
Youth in Government
throughout the year
• She can be reached at
850.245.9317
AlexFahnlander@fldoe.org
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
2
National Assessment of Educational
Progress (NAEP) 2015 Results
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
NAEP 2015 Results
• 2015 NAEP Mathematics and Reading assessment
results will probably be released the first week in
November
• National, State, and TUDA results will all be released
together - http://www.nationsreportcard.gov/
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
4
NAEP 2015 Results
• Results will also be posted on the NAEP webpage of
the FLDOE website. Best accessed through Chrome
http://www.fldoe.org/accountability/assessments/na
tional-international-assessments
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
5
Trends in International Mathematics
and Science Study (TIMSS) 2015 Results
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
TIMSS 2015 Results
• 2015 TIMSS Mathematics and Science assessment
results will be released December 2016
• International and Florida results will be released
together - https://nces.ed.gov/TIMSS/
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
7
2011 TIMSS Mathematics Results
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
8
2011 TIMSS Science Results
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
9
Summary of change in average U.S. TIMSS
scores over time
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
10
Program for International Student
Assessment (PISA) 2015
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
PISA 2015 in Florida
• Oct 5 – Nov 13, 2015; Results will be released
December 2016
• PISA testing in Florida will occur in 11 schools from 8
districts:
•
•
•
•
•
•
•
•
Duval
Hernando
Hillsborough
Manatee
Martin
Miami-Dade
Orange
Volusia
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
12
What is PISA?
• International assessment of 15year-old students conducted every
3 years since 2000
http://nces.ed.gov/surveys/pisa
• Measures how well students apply
their knowledge and skills in
solving problems in science,
reading, mathematics, financial
literacy, and collaborative
problem solving
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
PISA 2015 in Florida, con’t
• Most students are assessed in only 2 of the 5
assessment areas. No student takes every PISA
test question.
• 2015 Focus is Science
• 2012 Focus was Mathematics—Florida was oversampled to provide state-level results
• Students from more than 70 countries and
education systems have previously participated
• Sample size is 42 students – each student will
receive a certificate for 4 hours of volunteer service
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
PISA 2015 in Florida, con’t
• PISA 2015 will be administered on computers
provided by Westat
• PISA takes approximately 3 hours–includes instructions,
the assessment, and a brief survey questioning students
about themselves.
• Some students will also be selected to participate in an
additional 1-hour assessment in financial literacy that is
held after main PISA.
• Many PISA items are interactive. Sample questions
can be found at
http://nces.ed.gov/surveys/pisa/items_cba.asp
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
2012 Mathematics Literacy - U.S. 15-year-olds below OECD average
FL 467 (*,**)
MA 514 (*,**)
* = State avg. different from U.S.
** = State avg. different from OECD
CT 506 (*)
Average is higher than U.S.
average
Average not measurably different
from U.S. average
Average is lower than U.S.
average
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
16
2012 Science Literacy - U.S. 15-year-old students not measurably
different from OECD average
FL 485 (**)
MA 527 (*,**)
CT 521 (*,**)
* = State avg. different from U.S.
** = State avg. different from OECD
Average is higher than U.S.
average
Average not measurably different
from U.S. average
Average is lower than U.S.
average
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
17
2012 Reading Literacy - U.S. 15-year-olds not measurably
different from OECD average
MA 527 (*,**)
FL 492
CT 521 (*,**)
* = State avg. different than U.S.
** = State avg. different than OECD
Average is higher than U.S.
average
Average not measurably different
from U.S. average
Average is lower than U.S.
average
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
18
State Results for PISA 2012
Average Scores Compared to 65 OECD
Countries and Education Systems
• can be found at
http://nces.ed.gov/surveys/pisa/items_cba.asp
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
National Assessment of Educational
Progress (NAEP) 2016
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
20
NAEP 2016 in Florida
• Jan 25 – March 4, 2016
• National Results: Arts in March 2017
• Florida - 74 schools in 14 districts
•
•
•
•
•
Broward
Escambia
Hillsborough
Lake
Manatee
•
•
•
•
•
Miami-Dade
Orange
Osceola
Palm Beach
Pasco
•
•
•
•
Pinellas
Santa Rosa
Sarasota
Seminole
• Distribution of Assessments:
•
•
•
•
Grade 4: 16 schools, DBA, Mathematics and Reading
Grade 8: 31 schools, DBA, Mathematics and Reading or Writing
Grade 8: 10 schools, Paper-and-Pencil, Arts (Music and Visual Arts)
Grade 12: 17 schools, DBA, Writing
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
21
NAEP 2016 in Florida, con’t
Two types of assessments
1. Paper and Pencil Assessment
• Subject: Art
• Grade: 8
2. Digital-Based Assessment (DBA)
Microsoft Office Pro 3 tablets provided by Westat
• Subjects: Mathematics, Reading, and Writing
• Grades 4 and 8: Mathematics and Reading
• Grades 8 and 12: Writing
• Grades 4 and 8: Two administrations of 25 students
each
• Grade 12: One administration of 25 students
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
22
NAEP 2016, con’t
• DACs will receive a folder for each sampled school that
will include:
• School’s MyNAEP ID Number and assessment date
• Directions for registering on MyNAEP and completing the Provide School
Information (PSI) form—note: enrollment number is to be for the grade,
not the school
• An Overview of NAEP
• NAEP in Your School
• NAEP Tools on the Web
• Strategies for Motivating Grade 8 Students
DACs will receive a District MyNAEP ID number
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
23
Motivating Grade 8 Students
Student Brochure
School NAEP Coordinator Brochure
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
24
NAEP Pre-Assessment Activities
• Principals have already been given their
assessment dates by you
• Their next important task is to appoint a School
NAEP Coordinator (SNC)
• List of sampled students will be provided via the
MyNAEP website in early December
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
25
NAEP Pre-Assessment Activities
• In December a Westat staff person will call each of the
schools to
• Assist with registering for MyNAEP
• Review all MyNAEP “Prepare for Assessment Tasks”
o Go through the “Complete the Following:” tasks and
provide a short description of each
o Introduce the video tutorials
o Provide timeline for completing tasks
• Schedule the Preassessment Review Call (PRC) for January
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
NAEP Pre-Assessment Activities, cont.
• SNCs or DACs are required to upload a current roster
of students enrolled in the sampled grade in January
• Enrollment will be open for the first 3 weeks in January
• Schools with early assessment dates will be requested to
upload their current rosters the week of January 4
• Schools with later assessment dates will be requested to
wait until the weeks of Jan 11 or 18
• SNC will be notified if any new students have been added
to the sample
•
NAEP Help Desk is being redesigned to better meet the
needs of the schools
• Hiring more agents
• More comprehensive agent training
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
27
NAEP Pre-Assessment Activities, con’t.
• On the MyNAEP website, the SNC or Principal will
have to review the list of sampled students to:
• Identify withdrawn or ineligible students
• Update demographic data and SD/ELL classifications
• Indicate how each student identified as SD and/or ELL should be
assessed on NAEP (keep exclusions to a minimum, especially for
grade 8 ELL students scheduled for the Reading assessment)
• Indicate if a student has a Section 504 Plan and will need testing
accommodations
• Identify any student who is not enrolled in current grade, is a
foreign exchange student, or does not attend classes on campus
• Field representatives will make in-person visits to
schools experiencing difficulty with MyNAEP tasks
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
NAEP Pre-Assessment Activities, cont.
• Copy of Parent/Guardian Notification Letters no
longer collected by DACs
• On the MyNAEP website, SNCs will verify
parent/guardian notification at least 2 weeks prior to
the assessment
• School and Teacher Questionnaires will be
completed online
• SNC will send questionnaires via email to school
principal (or a designee) and to teachers of students
in the NAEP sample
• SNC will track the completion of the questionnaires
and be able to send reminder emails, as necessary
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
29
NAEP Pre-Assessment Activities, cont.
SNC will finalize assessment arrangements
• Provide start times and testing locations
• Provide 2 possible dates for make-up sessions
• Assessment cancelled due to weather
• Participation is less than 90%
SNC will
• Print and distribute
• Student Appointment Cards
• Teacher Notification Letter – date, time, location of
sessions
• List of Sampled Students
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
30
NAEP 2016
• Each student takes NAEP in only 1 of the 2
different formats (P&P or DBA) and in only 1 of
the 4 subject areas (Art, Mathematics, Reading,
or Writing)
• NAEP representatives will bring all necessary
materials and equipment to the schools
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
31
NAEP 2016
http://www.fldoe.org/accountability/assessments/nationalinternational-assessments/naep/resources-for-participatingschools.stml
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
32
NAEP 2017 and Beyond
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
33
NAEP 2017 - 2020
National Assessment Governing Board (NAGB) Goal:
NAEP 2017 operational mathematics and reading assessments will be completely
digital-based for the first time (http://www.nagb.org/). Grade 12 state-level NAEP
has been eliminated (administered in 2009 and 2013 in 13 self-selected states
including Florida). An * indicates state-level results will be available. Current
schedule adopted by NAGB March 2015
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
34
Progress in International Reading
Literacy Study (PIRLS) 2016
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
35
PIRLS and ePIRLS 2016
• March 1 – May 27, 2016; Results will be available
December 2017
• 8 schools in 6 districts
• Clay
• Duval
• Orange
• Palm Beach
• Seminole
• St. Lucie
• International assessment designed to measure trends in
reading achievement at the 4th grade level every 5 years
since 2001 https://surveys.nces.ed.gov/Pirls/
• Involves students from more than 40 countries
• U.S. Students from 176 schools in 42 states will be
sampled in 2016
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
36
PIRLS and ePIRLS 2016, con’t
• PIRLS will send a letter to school principals
on September 21st reminding them to name a
school coordinator if they haven’t already done so.
• PIRLS will contact schools again after October 5th to
• Obtain the name of the school coordinator (if not
already named)
• Determine data collection logistics
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
PIRLS and ePIRLS 2016, con’t
• Students (one or two classes of approximately 1824 students in each school) will participate in
PIRLS and ePIRLS on two successive days
• PIRLS is a paper-and-pencil assessment that
includes reading passages containing
informational and literary texts followed by openended and multiple-choice questions about the
texts – 2.5 hours
• ePIRLS uses a simulated Internet environment to
measure fourth-grade students’ achievement in
reading for informational purposes – 2.5 hours
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
38
PIRLS and ePIRLS 2016, con’t
• ePIRLS is to be conducted on school computers if
possible
• RTI International will provide laptop computers if:
• School computers are not compatible with ePIRLS
• School computers are not available at the school
• The school computer lab does not have enough
computers for each student
• RTI will work with participating schools to
coordinate logistics between October 2015 and
February 2016
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
39
Florida scored 2nd in the WORLD on
Grade 4 PIRLS in 2011!
Not significantly
different from
Florida
Not significantly
different from
U.S.
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
40
PIRLS 2016
http://www.fldoe.org/accountability/assessments
/national-international-assessments/pirls.stml
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
41
Data and Released Questions
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
42
NAEP Data Explorer (NDE)
http://nces.ed.gov/nationsreportcard/naepdata/
• Analyzes NAEP data
• Creates statistical tables,
graphics, and maps
• Compares the
performance of
jurisdictions and evaluates
trends over time
• Shows possible
relationships between
variables and student
performance
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
43
NAEP Questions Tool (NQT)
http://nces.ed.gov/nationsreportcard/itmrls
• Contains over 1,000 released
items from different content
areas
• Sorts items by domains,
objectives, cognitive ability,
and difficulty level
• Includes multiple-choice and
both short- and extendedresponse items
• Reports student
performance on a specific
test item by states and
subgroups
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
44
Questions Tools
International Data Explorer
PIRLS, PISA and TIMSS have released
items from mathematics, reading, and
science:
http://nces.ed.gov/surveys/internation
al/
Analyzes PIRLS, PISA and TIMSS data and
can create statistical tables, graphics and
maps:
http://nces.ed.gov/surveys/international
/ide/
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
Goodbye!
After serving as your NAEP State Coordinator since January 2003, I am
preparing to retire next July. That will make me one of the longest-serving
NSCs in the country. Many thanks to all of you for being such outstanding
colleagues over the years! I couldn’t have done the job without your help
and support!
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
46
www.FLDOE.org
Michele.Sonnenfeld@fldoe.org
http://www.fldoe.org/accountability/assessments/nationalinternational-assessments/naep
(850) 245-0787
www.FLDOE.org
© 2014, Florida Department of Education. All Rights Reserved.
47