NAEP
GUIDE TO NAEP RESULTS
TOOLKIT
WEST VIRGINIA
NATIONAL ASSESSMENT
OF EDUCATIONAL
PROGRESS
OVERVIEW OF NAEP
The National Assessment of Educational Progress (NAEP), first administered in 1969, is the largest
continuing and nationally representative assessment of what our nation’s students know and can
do in subjects such as mathematics, reading, science, and writing. NAEP is a congressionally
mandated project administered by the National Center for Education Statistics (NCES), within the
U.S. Department of Education and the Institute of Education Sciences (IES). The Commissioner of
Education Statistics is responsible for carrying out the NAEP project. The National Assessment
Governing Board oversees and sets policy for NAEP.
Teachers, principals, parents, policymakers, and researchers all use NAEP results to assess progress
and develop ways to improve education in the United States. The results of NAEP are released as
The Nation’s Report Card, and are available for the nation, states, and in some cases, urban districts.
No district in West Virginia meets the enrollment requirement to participate as a district. Typically,
comparison of student performance across states uses NAEP data, even though NAEP is not
generally mentioned in the report.
NAEP Participation
The 2001 Elementary and Secondary Education Act (ESEA), also known as “No Child Left Behind,”
requires state participation in NAEP in grades four and eight in reading and mathematics every other
year (i.e. odd numbered years). WV code requires participation in NAEP for “academic areas at the
various grades designated by the national assessment of educational progress officials.” (WV Code
§18-2E-2)
NAEP uses a carefully designed sampling procedure for the assessment to be representative of
schools and students in the state. First, schools are selected to be representative of schools in the
state. Then within each school, students are randomly selected to participate. NAEP typically selects
2000-3000 students in approximately 100 schools in each state for each grade and subject. Ideally for
West Virginia, each school sampled for NAEP would represent 1% of the student population for that
grade.
Student Confidentiality
It is against federal law to identify any student participating in the assessment. Since the first NAEP
assessment in 1969, student names have been kept completely confidential. After students complete
the assessment, their names are physically removed from the booklets and never leave their schools.
Instead of reporting individual scores, NAEP combines all student responses to provide information on
the performance of demographic groups of students. NAEP reports overall results for the nation, the
states, and for groups of students, such as males and females.
NAEP does not calculate scores for individual students, so neither the government nor the child’s
school or teacher will ever know how the child performed on NAEP.
1
NAEP
GUIDE TO NAEP RESULTS
TOOLKIT
WEST VIRGINIA
NATIONAL ASSESSMENT
OF EDUCATIONAL
PROGRESS
Administration of NAEP
The NAEP assessment is divided into 25-minute segments. The segments are distributed among the
entire assessment system with each student taking only 2 sections. All segments of the assessment
are compiled to report state level results. Since several student booklets combine to form the entire
assessment, results are reported at the state level and cannot be reported at the student or school
level.
NAEP assessments include both multiple-choice and open-ended responses. Open-ended response
questions are divided into short constructed responses and extended constructed response. A
segment with an extended constructed response item will have fewer questions.
OVERVIEW OF NAEP DATA
While NAEP provides a complex array of reports, analyses, and tools, the underlying data and
statistics can be organized into a succinct structure and hierarchy. Knowing what one can do and
cannot do with NAEP data allows for a more efficient approach to conducting analyses and answering
questions about the results.
NAEP data detail three basic educational areas involving students: achievement, demographics,
and inclusion. NAEP provides national, state, and district achievement information about student
knowledge and learning in various subject areas with the predominate ones being mathematics and
reading. NAEP also provides demographic information about specific groups of students (e.g. race/
ethnicity, gender, and school location) and information from questionnaires probing student, teacher,
and school characteristics. In addition, data are available regarding student inclusion in NAEP. These
data provide details for students with disabilities and English language learners (ELL), including rates
of identification, inclusion, exclusion, and accommodation. Figure 1 illustrates the organization of
NAEP data.
NAEP Data
Achievement
Data
Demographic
Data
Inclusion
Data
Figure 1
NAEP data are publicly reported via the Nation’s Report Card website (www.nationsreportcard.gov)
and the NAEP Data Explorer (http://nces.ed.gov/nationsreportcard/naepdata/).
2
NAEP
GUIDE TO NAEP RESULTS
TOOLKIT
WEST VIRGINIA
NATIONAL ASSESSMENT
OF EDUCATIONAL
PROGRESS
NAEP provides national, state and district achievement information about student knowledge and
learning in various subject areas with the predominant ones being mathematics and reading. The
foundational unit of reporting is the student. For example, responses from teacher questionnaires are
reported as “percent of students who have teachers who” answered the question a certain way. Table
1 shows examples of questionnaire questions and the interpretation of the data.
Table 1: Questionnaire Comparison
Questionnaire
Question Text
Result Interpretation
Statement
(using 2013 grade 4 mathematics)
School
Grade 4
Student
Grade 4
Mathematics
About what percentage of your students
is absent on an average day? (Include
excused and unexcused absences in
calculating this rate)
Choices: 0-2%, 3-5%, 6-10%, More than
10%
How many days were you absent from
school in the last month?
Choices: None, 1 or 2 days, 3 or 4 days, 5
to 10 days, More than 10 days
54% of West Virginia students
attend schools reporting a 3-5%
absent rate on an average day.
34% of West Virginia 4th grade
students reported being absent
from school 1 or 2 days last month.
NAEP Achievement Data
NAEP achievement data are reported in two different metrics: (1) scale scores and (2) achievement
levels.
Scale scores define a range of scores on which student achievement will be reported. For example,
NAEP mathematics and reading are reported on a scale of 0 to 500 while NAEP science and writing
are reported on a scale of 0 to 300. Scale scores are continuous in nature (i.e. numeric; can be
thought of as numbers). Comparisons cannot be made between different scales. For example, scores
from mathematics cannot be compared to those of reading.
Achievement levels are defined by the National Assessment Governing Board (www.nagb.
org) and are used in reporting student performance as Advanced, Proficient, Basic, and Below
Basic. While NAEP proficiency levels may seem to mirror those of state assessments, they are set
independently of state assessment programs and do not necessarily align with states’ achievement
levels. Achievement levels are categorical in nature and are summarized using percentages. They are
summarized as the percent of students at an achievement level (referred to as discrete) or the percent
of students at or above an achievement level (referred to as cumulative). Figure 2 expands on the
NAEP achievement data.
3
NAEP
GUIDE TO NAEP RESULTS
TOOLKIT
WEST VIRGINIA
NATIONAL ASSESSMENT
OF EDUCATIONAL
PROGRESS
Averages
“Average”
“Mean”
Percentiles
“10th”; “25th”; “50th”;
“75th”; “90th”
Percentages
“Percent at”;
“Percent at or above”
Scale Score
NAEP
Achievement
Data
Achievement
Levels
Figure 2
NAEP Demographic Data
NAEP demographic data are reported for two different compositions: (1) subgroup composition and
(2) achievement level composition. Reporting of NAEP demographic data solely uses one or the
other of these breakdowns and represents student membership in specific groups. Both subgroup
composition and achievement level composition are summarized using percentages and reflect group
membership. Figure 3 illustrates the breakdown of NAEP achievement data.
NAEP
Demographic
Data
Subgroup
Composition
Achievement
Level
Composition
Percentage
Percentage
Figure 3
Subgroup composition defines student membership in specific groups, such as eligibility for
the National School Lunch Program, race/ethnicity, gender, etc. One may think of these variables as
characteristics of students. Students can belong to one and only one subgroup for each group type.
Subgroups are also defined through data collected from student, teacher, and school questionnaires.
Subgroup composition reporting uses percentages of students in each subgroup. For example, the
table below shows the subgroup composition from the 2013 grade 4 mathematics assessment for the
question “About what percentage of your students is absent on an average day?”
4
NAEP
GUIDE TO NAEP RESULTS
TOOLKIT
WEST VIRGINIA
NATIONAL ASSESSMENT
OF EDUCATIONAL
PROGRESS
Table 2
About what percentage of your students is
absent on an average day?
Percentage
0-2%
3-5%
6-10%
27
54
19
More than
10%
#
# Rounds to zero.
Teacher and school questionnaires define student characteristics for reporting. However, it is
extremely important to understand that no inferences are made to the teacher or schools. Inferences
are to the student level, not teacher or school level.
Achievement level composition started in the 2009 reporting cycle with the release of the
redesigned NAEP Data Explorer and basically describes the composition of subgroups within
achievement levels. For example, it can provide detail of the percentage breakdown of lunch status
within the Basic achievement level.
Table 3
About what percentage of your students is
absent on an average day?
Below Basic
At Basic
At Proficient
At Advanced
0-2%
3-5%
6-10%
15
45
34
6
19
44
32
5
25
47
26
3
More than
10%
‡
‡
‡
‡
‡ means reporting standard not met
Note: In Table 2, since the percentage of students have schools which reported more than 10% of the
students were absent on an average day rounds to zero, the reporting standard is not met for any of
the achievement levels.
Inclusion Data
It is important for NAEP to assess as many of the students selected to participate as is possible.
Assessing representative samples of students, including students with disabilities (SD) and English
language learners helps to ensure that NAEP results accurately reflect the educational performance
of all students in the target population and can continue to serve as a meaningful measure of U.S.
students’ academic achievement over time.
5
NAEP
GUIDE TO NAEP RESULTS
TOOLKIT
WEST VIRGINIA
NATIONAL ASSESSMENT
OF EDUCATIONAL
PROGRESS
The National Assessment Governing Board, which sets policy for NAEP, has been exploring ways to
ensure that NAEP continues to appropriately and consistently include as many studetns as possible
for all jurisdictions assessed and reported. In March 2010, the Governing Board adopted a new
policy, NAEP Testing and Reporting on Students with Disabilities and English Language Learners.
This policy was the culmination of work with experts in testing and curriculum, and those who work
with exceptional children and students learning to speak English. The policy aims to
»» Maximize participation of sampled students in NAEP,
»» Reduce variation exclusion rates for SD and ELL students across states and districts,
»» Develop uniform national rules for including students in NAEP, and
»» Ensure that NAEP is fully representative of SD and ELL students.
Determining whether each jurisdiction has met the NAEP inclusion goals involves looking at three
different inclusion rates—an overall inclusion rate, an inclusion rate for SD students, and an inclusion
rate for ELL students. Each inclusion rate is calculated as the percentage of sampled students who
were included in the assessment.
Unlike achievement data and demographic data, inclusion data is not found in the NAEP Data
Explorer. You can access it in the data tables available on the Nation’s Report Card site
(www.nationsreportcard.gov). When referencing NAEP inclusion data, extreme caution should be
used. The tables look very much alike, so the table titles are critical to the correct interpretation of the
results.
NAEP inclusion data are only reported as a percentage of one of four groups: all sampled students,
SD and/or ELL students, SD students, or ELL students. Inclusion data contain identification rate,
inclusion rate, exclusion rate, and accommodation rates for each of the reported groups. See Figure 4.
Identification
% of all
students
Inclusion
Exclusion
Accommodation
Identification
% SD and/or
ELL students
NAEP
Inclusion
Data
Inclusion
Exclusion
Accommodation
Identification
% of SD
students
Inclusion
Exclusion
Accommodation
Identification
% of ELL
students
Inclusion
Exclusion
Accommodation
Figure 4
6